AWS Full Course In 11 Hours | AWS Tutorial For Beginners | AWS Training For Beginners | Simplilearn

[Music] hey everyone welcome to this video on aws full course we'll begin this full course video with what aws is and how aws services work and we will be looking at a bunch of aws services like ec2 lambda s3 im cloud formation ecs route 53 elastic beam stock vpc sagemaker cloud front auto scaling and redshift after this we will do a comparison of popular cloud providers that is aws azure and gcp moving further we will see how kubernetes is implemented on aws and how to become a solution architect we will also guide you through aws certification exams and in which we have explained how to choose the right certification for your career we will then conclude this video by discussing the essential interview questions and answers to help every individual crack an interview by the end of this video i can assure you that all your aws career related queries would have been answered for this training with me i have our experienced aws specialist sam and rahul together we will take you through the various topics of aws so let's start with an exciting video on aws full course but before we begin make sure to subscribe to our youtube channel and hit the bell icon to never miss an update from simplylearn meet rob he runs an online shopping portal the portal started with a modest number of users but has recently been seeing a surge in the number of visitors on black friday and other holidays the portal saw so many visitors that the servers were unable to handle the traffic and crashed is there a way to improve performance without having to invest in a new server wondered rob a way to upscale or downscale capacity depending on the number of users visiting the website at any given point well there is amazon web services one of the leaders in the cloud computing market before we see how aws can solve rob's problem let's have a look at how aws reach the position it is at now aws was first introduced in 2002 as a means to provide tools and services to developers to incorporate features of amazon.com to their website in 2006 its first cloud services offering was introduced in 2016 aws surpassed its 10 billion revenue target and now aws offers more than 100 cloud services that span a wide range of domains thanks to this the aws cloud service platform is now used by more than 45 of the global market now let's talk about what is aws aws or amazon web service is a secure cloud computing platform that provides computing power database networking content storage and much more the platform also works with a pay as you go pricing model which means you only pay for how much of the services offered by aws you use some of the other advantages of aws are security aws provides a secure and durable platform that offers end-to-end privacy and security experience you can benefit from the infrastructure management practices born from amazon's years of experience flexible it allows users to select the os language database and other services easy to use users can host applications quickly and securely scalable depending on user requirements applications can be scaled up or down aws provides a wide range of services across various domains what if rob wanted to create an application for his online portal aws provides compute services that can support the app development process from start to finish from developing deploying running to scaling the application up or down based on the requirements the popular services include ec2 aws lambda amazon light cell and elastic bean stock for storing website data rob could use aws storage services that would enable him to store access govern and analyze data to ensure that costs are reduced agility is improved and innovation accelerated popular services within this domain include amazon s3 ebs s3 glacier and elastic file storage rob can also store the user data in a database with aw services which he can then optimize and manage popular services in this domain include amazon rds dynamodb and redshift if rob's businesses took off and he wanted to separate his cloud infrastructure or scale up his work requests and much more he would be able to do so with the networking services provided by aws some of the popular networking services include amazon vpc amazon route 53 and elastic load balancing other domains that aws provide services in are analytics blockchain containers machine learning internet of things and so on and there you go that's aws for you in a nutshell now before we're done let's have a look at a quiz which of these services are incorrectly matched one two three four we'll be pinning the question in the comment section comment below with your answer and stand a chance to win an amazon voucher several companies around the world have found great success with aws companies like netflix twitch linkedin facebook and bbc have taken advantage of the services offered by aws to improve their business efficiency and thanks to their widespread usage aws professionals are in high demand they're highly paid and earn up to more than 127 thousand dollars per annum once you're aws certified you could be one of them too hello everyone let me introduce myself as sam a multi-platform cloud architect and trainer and i'm so glad and i'm equally excited to talk and walk you through the session about what aws is and talk to you about some services and offerings and about how companies get benefited by migrating their applications and infra into aws so what's aws let's talk about that now before that let's talk about how life was without any cloud provider and in this case how life was without aws so let's walk back and picture how things were back in 2000 which is not so long ago but a lot of changes a lot of changes for better had happened since that time now back in 2000 a request for a new server is not an happy thing at all because a lot of money a lot of validations a lot of planning are involved in getting a server online or up and running and even after we finally got the server it's not all said and done a lot of optimization that needs to be done on that server to make it worth it and get a good return on investment from that server and even after we have optimized for a good return on investment the work is still not done there will often be a frequent increase and decrease in the capacity and you know even news about our website getting popular and getting more hits it's still an bittersweet experience because now i need to add more servers to the environment which means that it's going to cost me even more but thanks to the present day cloud technology if the same situation were to happen today my new server it's almost ready and it's ready instantaneously and with the swift tools and technologies that amazon is providing uh in provisioning my server instantaneously and adding any type of workload on top of it and making my storage and server secure you know creating a durable storage where data that i store in the cloud never gets lost with all that features amazon has got our back so let's talk about what is aws there are a lot of definitions for it but i'm going to put together a simple and a precise definition as much as possible now let me iron that out cloud still runs on and hardware all right and there are certain features in that infrastructure in that cloud infrastructure that makes cloud cloud or that makes aws a cloud provider now we get all the services all the technologies all the features and all the benefits that we get in our local data center like you know security and compute capacity and databases and in fact you know we get even more cool features like content caching in various global locations around the planet but again out of all the features the best part is that i get or we get everything on a pay as we go model the less i use the less i pay and the more i use the less i pay per unit very attractive isn't it right and that's not all the applications that we provision in aws are very reliable because they run on a reliable infrastructure and it's very scalable because it runs on an on-demand infrastructure and it's very flexible because of the designs and because of the design options available for me in the cloud let's talk about how all this happened aws was launched in 2002 after the amazon we know as the online retail store wanted to sell their remaining or unused infrastructure as a service or as an offering for customers to buy and use it from them you know sell infrastructure as a service the idea sort of clicked and aws launched their first product first product in 2006 that's like four years after the idea launch and in 2012 they held a big sized customer even to gather inputs and concerns from customers and they were very dedicated in making those requests happen and that habit is still being followed it's still being followed as a reinvent by aws and at 2015 amazon announced its revenue to be 4.6 billion and in 2015 through 2016 aws launched products and services that helped migrate customer services into aws well there were products even before but this is when a lot of focus was given on developing migrating services and in the same year that's in 2016 amazon's revenue was 10 billion and not but not the least as we speak amazon has more than 100 products and services available for customers and get benefited from all right let's talk about the services that are available in amazon let's start with this product called s3 now s3 is a great tool for internet backup and it's it's the cheapest storage option in the object storage category and not only that the data that we put in s3 is retrievable from the internet s3 is really cool and we have other products like migration and data collection and data transfer products and here we can not only collect data seamlessly but also in a real time way monitor the data or analyze the data that's being received that they're cool products like aws data transfers available that helps achieve that and then we have products like ec2 elastic compute cloud that's an resizable computer where we can anytime anytime after the size of the computer based on the need or based on the forecast then we have simple notification services systems and tools available in amazon to update us with notifications through email or through sms now anything anything can be sent through email or through sms if you use that service it could be alarms or it could be service notifications if you want stuff like that and then we have some security tools like kms key management system which uses aes 256 bit encryption to encrypt our data at rest then we have lambda a service for which we pay only for the time in seconds seconds it takes to execute our code and uh we're not paying for the infrastructure here it's just the seconds the program is going to take to execute the code for the short program we'll be paying in milliseconds if it's a bit bigger program we'll be probably paying in 60 seconds or 120 seconds but that's not cheap lot simple and lots cost effective as against paying for service on an ugly basis which a lot of other services are well that's cheap but using lambda is a lot cheaper than that and then we have services like route 53 a dns service in the cloud and now i do not have to maintain a dns account somewhere else and my cloud environment with aws i can get both in the same place all right let me talk to you about um how aws makes life easier or how companies got benefited by using aws as their it provider for their applications or for the infrastructure now uniliver is a company and they had a problem right and they had a problem and they picked aws as a solution to their problem right now this company was sort of spread across 190 countries and they were relying on a lot of digital marketing for promoting their products and their existing environment their legacy local environment proved not to support their changing id demands and they could not standardize their old environment now they chose to move part of their applications to aws because they were not getting what they wanted in their local environment and since then you know rollouts were easy provisioning new applications became easy and even provisioning infrastructure became easy and they were able to do all that in push button scaling and needless to talk about backups that are safe and backups that can be securely accessed from the cloud as needed now that company is growing along with aws because of their swift speed in rolling out deployments and being able to access secure backups from various places and generate reports and in fact useful reports out of it that helps their business now on the same lines let me also talk to you about kellogg's and how they got benefited by using amazon now kellogg's had a different problem it's one of its kind now their business model was very dependent on an infrared that will help to analyze data really fast right because they were running promotions based on the analyzed data that they get so they're being able to respond to the analyzed data as soon as possible was critical or vital in their environment and luckily sap running on hana environment is what they needed and uh you know they picked that service in the cloud and that's sort of solved the problem now the company does not have to deal with uh maintaining their legacy infra and maintaining their heavy compute capacity and maintaining their database locally all that is now moved to the cloud or they are using cloud as their i.t service provider and and now they have a greater and powerful ita environment that very much complements their business let me start the session with this scenario let's imagine how life would have been without spotify for those who are hearing about spotify for the first time spotify is an online music service offering and it offers instant access to over 16 million licensed songs spotify now uses aws cloud to store the data and share it with their customers but prior to aws they had some issues imagine using spotify before aws let's talk about that back then users were often getting errors because spotify could not keep up with the increased demand for storage every new day and that led to users getting upset and users cancelling the subscription the problem spotify was facing at that time was their users were present globally and were accessing it from everywhere and they had different latency in their applications and spotify had a demanding situation where they need to frequently catalog the songs released yesterday today and in the future and this was changing every new day and the songs coming in rate was about 20 000 a day and back then they could not keep up with this requirement and needless to say they were badly looking for a way to solve this problem and that's when they got introduced to aws and it was a perfect fit and match for their problem aws offered at dynamically increasing storage and that's what they needed aws also offered tools and techniques like storage lifecycle management and trusted advisor to properly utilize the resource so we always get the best out of the resource used aws address their concerns about easily being able to scale yes you can scale the aws environment very easily how easily one might ask it's just a few button clicks and aws solved spotify's problem let's talk about how it can help you with your organization's problem let's talk about what is aws first and then let's bleed into how aws became so successful and the different types of services that aws provides and what's the future of cloud and aws in specific let's talk about that and finally we'll talk about a use case where you will see how easy it is to create a web application with aws all right let's talk about what is aws aws or amazon web services is a secure cloud service platform it is also pay as you go type billing model where there is no upfront or capital cost we'll talk about how soon the service will be available well the service will be available in a matter of seconds with aws you can also do identity and access management that is authenticating and authorizing a user or a program on the fly and almost all the services are available on demand and most of them are available instantaneously and as we speak amazon offers 100 plus services and this list is growing every new week now that would make you wonder how aws became so successful of course it's their customers let's talk about the list of well-known companies that has their idea environment in aws adobe adobe uses aws to provide multi-terabyte operating environments for its customers by integrating its system with aws cloud adobe can focus on deploying and operating its own software instead of trying to you know deploy and manage the infrastructure airbnb is another company it's an community marketplace that allows property owners and travelers to connect each other for the purpose of renting unique vacation spaces around the world and the airbnb community users activities are conducted on the website and through iphones and android applications airbnb has a huge infrastructure in aws and they're almost using all the services in aws and are getting benefited from it another example would be autodesk autodesk develops software for engineering designing and entertainment industries using services like amazon rds or rational database servers and amazon s3 or amazon simple storage servers autodesk can focus on deploying or developing its machine learning tools instead of spending that time on managing the infrastructure aol or american online uses aws and using aws they have been able to close data centers and decommission about 14 000 in-house and co-located servers and move mission critical workload to the cloud and extend its global reach and save millions of dollars on energy resources bitdefender is an internet security software firm and their portfolio of software's include antivirus and anti-spyware products bitdefender uses ec2 and they're currently running few hundred instances that handle about five terabytes of data and they also use elastic load balancer to load balance the connection coming in to those instances across availability zones and they provide seamless global delivery of servers because of that the bmw group it uses aws for its new connected car application that collects sensor data from bmw 7 series cars to give drivers dynamically updated map information canon's office imaging products division benefits from faster deployment times lower cost and global reach by using aws to deliver cloud-based services such as mobile print the office imaging products division uses aws such as amazon s3 and amazon route 53 amazon cloudfront and amazon im for their testing development and production services comcast it's the world's largest cable company and the leading provider of internet service in the united states comcast uses aws in a hybrid environment out of all the other cloud providers comcast chose aws for its flexibility and scalable hybrid infrastructure docker is a company that's helping redefine the way developers build ship and run applications this company focuses on making use of containers for this purpose and in aws the service called the amazon ec2 container service is helping them achieve it the esa or european space agency although much of esa's work is done by satellites some of the programs data storage and computing infrastructure is built on amazon web services esa chose aws because of its economical pay as ego system as well as its quick startup time the guardian newspaper uses aws and it uses a wide range of aws services including amazon kinesis amazon redshift that power an analytic dashboard which editors use to see how stories are trending in real time financial times ft is one of the world's largest leading business news organization and they used amazon redshift to perform their analysis a funny thing happened amazon redshirt performed so quickly that some analysis thought it was malfunctioning they were used to running queries overnight and they found that the results were indeed correct just as much faster by using amazon redshift fd is supporting the same business functions with costs that are 80 percentage lower than what was before general electric ge is at the moment as we speak migrating more than 9000 workloads including 300 desperate erp systems to aws while reducing its data center footprint from 34 to 4 over the next three years similarly howard medical school htc imdb mcdonald's nasa kellogg's and lot more are using the services amazon provides and are getting benefited from it and this huge success and customer portfolio is just the tip of the iceberg and if we think why so many adapt aws and if we let aws answer that question this is what aws would say people are adapting aws because of the security and durability of the data and end-to-end privacy and encryption of the data and storage experience we can also rely on aws way of doing things by using the aws tools and techniques and suggested best practices built upon the years of experience it has gained flexibility there is a greater flexibility in aws that allows us to select the os language and database easy to use swiftness in deploying we can host our applications quickly in aws beat a new application or migrating an existing application into aws scalability the application can be easily scaled up or scaled down depending on the user requirement cost saving we only pay for the compute power storage and other resources you use and that too without any long-term commitments now let's talk about the different types of services that aws provides the services that we talk about fall in any of the following categories you see like you know compute storage database security customer engagement desktop and streaming machine learning developers tools stuff like that and if you do not see the service that you're looking for it's probably is because aws is creating it as we speak now let's look at some of them that are very commonly used within computer services we have amazon ec2 amazon elastic bean stock amazon light sale and amazon lambda amazon ec2 provides compute capacity in the cloud now this capacity is secure and it is resizable based on the user's requirement now look at this the requirement for the web traffic keeps changing and behind the scenes in the cloud ec2 can expand its environment to three instances and during no load it can shrink its environment to just one resource elastic beanstalk it helps us to scale and deploy web applications and it's made with a number of programming languages elastic beanstalk is also an easy to use service for deploying and scaling web applications and services deployed between java.net php node.js python ruby docker and lot other familiar services such as apache passenger and iis we can simply upload our code and elastic beanstalk automatically handles the deployment from capacity provisioning to load balancing to auto scaling to application health monitoring and amazon lightsail is a virtual private server which is easy to launch and easy to manage amazon lightsail is the easiest way to get started with aws for developers who just need a virtual private server lightsail includes everything you need to launch your project quickly on a virtual machine like ssd based storage a virtual machine tools for data transfer dns management and a static ip and that too for a very low and predictable price aws lambda has taken cloud computing services to a whole new level it allows us to pay only for the compute time no need for provisioning and managing servers and aws lambda is a compute service that lets us run code without provisioning or managing servers lambda executes your code only when needed and scales automatically from few requests per day to thousands per second you pay only for the compute time you consume there is no charge when your code is not running let's look at some storage services that amazon provides like amazon s3 amazon glacier amazon abs and amazon elastic file system amazon s3 is an object storage that can store and retrieve data from anywhere websites mobile apps iot sensors and so on can easily use amazon x3 to store and retrieve data it's an object storage built to store and destroy any amount of data from anywhere with its features like flexibility in managing data and the durability it provides and the security that it provides amazon simple storage service or s3 is a storage for the internet and glacier glacier is a cloud storage service that's used for archiving data and long-term backups and this glacier is an secure durable and extremely low-cost cloud storage service for data archiving and long term backups amazon ebs amazon elastic block store provides block store volumes for the instances of ec2 and this elastic block store is highly available and a reliable storage volume that can be attached to any running instance that is in the same availability zone abs volumes that are attached to the ec2 instances are exposed as storage volumes that persistent independently from the lifetime of the instance an amazon elastic file system or efs provides an elastic file storage which can be used with aws cloud service and resources that are on premises an amazon elastic file system it's an simple it's scalable it's an elastic file storage for use with amazon cloud services and for on-premises resources it's easy to use and offers a simple interface that allows you to create and configure file systems quickly and easily amazon file system is built to elastically scale on demand without disturbing the application growing and shrinking automatically as you add and remove files your application have the storage they need and when they need it now let's talk about databases the two major database flavors are amazon rds and amazon redshift amazon rds it really eases the process involved in setting up operating and scaling a relational database in the cloud amazon rds provides cost efficient and resistable capacity while automating time consuming administrative tasks such as hardware provisioning database setup patching and backups it sort of frees us from managing the hardware and sort of helps us to focus on the application it's also cost effective and resizable and it's also optimized for memory performance and input and output operations not only that it also automates most of the services like taking backups you know monitoring stuff like that it automates most of those services amazon redshift amazon redshift is a data warehousing service that enables users to analyze the data using sql and other business intelligent tools amazon reshift is an fast and fully managed data warehouse that makes it simple and cost effective analyze all your data using standard sql and your existing business intelligent tools it also allows you to run complex analytic queries against petabyte of structured data using sophisticated query optimizations and most of the results they generally come back in seconds all right let's quickly talk about some more services that aws offers there are a lot more services that aws provides but are we going to look at some more services that are widely used aws application discovery services help enterprise customers plan migration projects by gathering information about their on-premises data centers in a planning a data center migration can involve thousands of workloads they are often deeply interdependent server utilization data and dependency mapping are important early first step in migration process and this aws application discovery service collects and presents configuration usage and behavior data from your servers to help you better understand your workloads route 53 it's a network and content delivery service it's an highly available and scalable cloud domain name system or dns service and amazon route 53 is fully compliant with ipv6 as well elastic load balancing it's also a network and content delivery service elastic load balancing automatically distributes incoming application traffic across multiple targets such as amazon ec2 instance containers and ip addresses it can handle the varying load of your application traffic in a single availability zone and also across availability zones aws auto scaling it monitors your application and automatically adjusts the capacity to maintain steady and predictable performance at a lowest possible cost using aws auto scaling it's easy to set up application scaling for multiple resources across multiple services in minutes auto scaling can be applied to web services and also for db services aws identity and access management it enables you to manage access to aws services and resources securely using iam you can create and manage aws users and groups and use permissions to allow and deny their access to aws resources and moreover it's a free service now let's talk about the future of aws well let me tell you something cloud is here to stay here's what in store for aws in the future as years pass by we're going to have a variety of cloud applications born like iot artificial intelligence business intelligence serverless computing and so on cloud will also expand into other markets like healthcare banking space automated cars and so on as i was mentioning some time back lot or greater focus will be given to artificial intelligence and eventually because of the flexibility and advantage that cloud provides we're going to see a lot of companies moving into the cloud all right let's now talk about how easy it is to deploy and web application in the cloud so the scenario here is that our users like a product and we need to have a mechanism to receive input from them about their likes and dislikes and you know give them the appropriate product as per their need all right though the setup and the environment it sort of looks complicated we don't have to worry because aws has tools and technologies which can help us to achieve it now we're going to use services like route 53 services like cloudwatch ec2 s3 and lot more and all these put together are going to give an application that's fully functionable and an application that's going to receive the information like using the services like route 53 cloudwatch ec2 and s3 we're going to create an application and that's going to meet our need so back to our original requirement all i want is to deploy a web application for a product that keeps our users updated about the happenings and the new comings in the market and to fulfill this requirement here is all the services we would need ec2 here is used for provisioning the computational power needed for this application and ec2 has a vast variety of family and types that we can pick from for the types of workloads and also for the intents of the workloads we're also going to use s3 for storage and s3 provides any additional storage requirement for the resources or any additional storage requirement for the web applications and we are also going to use cloudwatch for monitoring the environment and cloudwatch monitors the application and the environment and it provides trigger for scaling in and scaling out the infrastructure and we're also going to use route 53 for dns and route 53 helps us to register the domain name for our web application and with all the tools and technologies together all of them put together we're going to make an application a perfect application that caters our need all right so i'm going to use elastic beanstalk for this project and the name of the application is going to be as you see gsg sign up and the environment name is gsg sign up environment one let me also pick a name let me see if this name is available yes that's available that's the domain name so let me pick that and the application that i have is going to run on node.js so let me pick that platform and launch now as you see elastic beanstalk this is going to launch an instance it's going to launch the monitoring setup or the monitoring environment it's going to create a load balancer as well and it's going to take care of all the security features needed for this application all right look at that i was able to go to that url which is what we gave and it's now having an default page shown up meaning all the dependencies for the software is installed and it's just waiting for me to upload the code or in specific the page required so let's do that let me upload the code i already have the code saved here that's my code and that's going to take some time all right it has done its thing and now if i go to the same url look at that i'm being thrown an advertisement page all right so if i sign up with my name email and stuff like that you know it's going to receive the information and it's going to send an email to the owner saying that somebody had subscribed to your service that's the default feature of this app look at that email to the owner saying that somebody had subscribed to your app and this is their email address stuff like that not only that it's also going to create an entry in the database and dynamodb is the service that this application users to store data there's my dynamodb and if i go to tables right and go to items i'm going to see that a user with name samuel and email address so and so has said okay or has shown interest in the preview of my site or product so this is where this is how i collect those information right and some more things about the infrastructure itself is it is running behind and load balancer look at that it had created a load balancer it had also created an auto scaling group now that's the feature of elastic load balancer that we have chosen it has created an auto scaling group and now let's put this url you see this it's not a fancy url right it's an amazon given url a dynamic url so let's put this url behind our dns let's do that so go to services go to route 53 go to hosted zone and there we can find the dns name right so that's a dns name all right all right let's create an entry and map that url to our load balancer right and create now technically if i go to this url it should take me to that application all right look at that i went to my custom url and now that's pointed to my application previously my application was having a random url and now it's having a custom url choose from over 300 in-demand skills and get access to 1 000 plus hours of video content for free visit skill up by simply learn click on the link in the description to know more let me help you understand about aws ec2 a compute servers in the cloud through this video somewhere far in the country a scientist has a lab far in the woods and at one day he stumbled on a calculation that he had trouble solving and he was very certain that his computer can handle it and he asked his computer to do the math or the calculation for him and the computer thought about it for a while then took couple of hours in trying to process the data and eventually at one point it gave up and died due to low memory and this old scientist is now clueless about how he was going to finish the task and complete his invention and let the whole world know about it and that's when he was introduced to this new friend aws as you see on the screen aws was very handy in replacing his old machines and it was able to give him the service that he wanted at that moment aws has been around for a while helping people with their computer needs and now this new scientist friend in need is welcomed by the new and trendy id technology aws and sure enough his new aws friend welcomed and walked him through the new way of computing and this new happy-faced aws friend talked to this scientist about aws ec2 in specific and walked him through about all the new innovations in it and in cloud that he had been missing all these days so his new friend aws started the conversation about explaining how there is no need for any hardware units no managing hardware or provisioning in aws and secondly it explained increasing and decreasing the capacity as per the demand is just a click away and the best part here is that there is no upfront commitment and we only pay for what we have used just like we pay electricity and water bills and all this comes with complete control from our side in other words the whole key for this infrastructure is with us and as if this was not enough all this comes with enterprise great security that's beyond imagination in on premises and if this still does not excite you then this definitely would you can work from anywhere in the world now this really made the scientist to get excited especially when he thought about working from home working from home there is nothing like it isn't it now this person the scientist is not tied up to a particular environment he can work from home work on the fly work from anywhere still the data and everything else in his id environment is secure and safe now let's move to the next level of discussion let's talk about what is ec2 and some of the use cases of ec2 and this use case we're going to talk about are this architecture we're going to talk about it notifies a group of users about a new letter and the components or resources this architecture would need the first ec2 instance then sns a simple notification service and coupling ec2 and s3 service together that's all it would require to get an architecture that notifies a group of users about a new newsletter and now i think it would be a good time to talk about what ec2 is aws offers plenty of services offered under different domains like you know compute storage database migration network management tools media services security business productivity application integration machine learning game development and lot more coming up out of which ec2 falls under the compute capacity so what is ec2 ec2 is a web service which aims to make life easier for developers for providing secure and resizable compute capacity in the cloud with ec2 it is very easy to scale up or scale down our infrastructure based on the demand and not only that this ec2 service can be integrated well with almost all the services in amazon and out of all this the best part could be we only pay for what we use all right let's talk about this use case a use case here is that a successful business owner has a bunch of users and a successful product that's running and now he has developed few more products that he thinks will be very successful that he thinks his customers are going to like now how can he advertise his product to his new and prospective customers or the solution for this question can be addressed by aws in aws we can use services like simple notification service sns and ec2 for compute and s3 for storage we can in a way integrate them all and achieve this business use case and that sort of got this business owner very cheered up and now he wants to use the service and he wants to get benefited from the service he wants to advertise or he wants to notify his users every time the company creates a newsletter all right so let's talk about what it would take to get this environment up and running or what it would take to connect the environment and put the applications on top of it firstly we would require an aws account and then for compute capacity we would require an ec2 instance and here's how we go about doing it the first thing is to create an ami which is amazon mission image that's really the softwares and the application packages we would need to run our application and the second is to choose the hardware in here it's the instance type depending on the workload we would be choosing the hardware and depending on the intents of the workload we will be choosing the size of the hardware and finally we would configure the instances and how many instance do i want you know which subnet do i want them in and what's going to be the in a stop or terminate a behavior of the instance and do i want to update any patches when the instance starts running all those pieces of information go in here when we configure the instance and then the first three steps is really about the os volume and the basic hardware now it's time to add additional storage to the ec2 instance that would be step four here we add additional storage to the ec2 instance and then tags we use tags or we would configure tags to easily identify an ec2 instance at a later point you know we give it some meaningful names so we can identify like you know which team it belongs to which billing department it belongs to what's the purpose behind launching this instance stuff like that in an environment where we run 700 to 800 or even more instances identifying an instance and trying to understand you know who owns the resource for what purpose we created it could be an full-time work so tagging comes to a rescue at that time after tagging as step 6 we would configure the firewall which is also called security group for the ec2 instant and this is where we would allow or deny connection from external world to this particular ec2 instance well it works both ways from outside and from inside out this firewall blocks the connection based on port number and ip address and finally as step 7 we review all the configurations that you have done and we make sure that the configurations is what we wanted and finally click on submit that's going to launch an ec2 instance all right this was just an overview of how to create an ec2 instance now let's talk about each and every step in detail so to begin with let's talk about how to create an ami well the ami is just a template a template that's used to create a new instance or an new computer or a new vm or a new machine based on the user requirement the things that go into an ami are the software the operating system the additional applications that get installed in it stuff like that the ami will also contain software information you know information about operating system information about access permission information about volumes they all compact in the ami again the ami is of two types one is predefined amis are called amazon provided amis the other one would be custom amis the amis that we create and if you're looking at a particular ami that you don't want to create but still want to get it from amazon there is a place or a portal called ami marketplace there we get like thousands of amis in there available for us to shop and use them on a pay as you go business model and use them as pay as you go billing so there you can search ami that you're looking for most probably you'll be able to get it there now let's talk about choosing the instance type the instance type is basically the hardware specification that's required for a machine that we're trying to build and the instant types is categorized into five main families they are to begin with it's compute optimized now compute optimize gives us lots of compute power or lots of processing power so if my application is going to require a lot of processing power i should be picking compute optimized instance and the second one is memory optimized now this is very good for application that require in-memory caching you know there are some application that performs well with cash or through cash or the application would create a lot of data that it wants to keep in cache for rereading or for processing you know for lengthy processing stuff like that for those type of application this memory optimized instance that comes with in memory cache is a very good use case and the third one is the instant that comes with the gpu otherwise called gpu optimized gpu stands for graphical process unit and this is very good for application that deals with gaming this is very good for application that's going to require large graphical requirements and storage optimize is the fourth option just like the name says this is a very good use case for storage servers and the fifth family type is general purpose just like the name says it's for general purpose if you're not particular about the family then you generally would end up picking the general purpose because here the services are sort of equally balanced you know you'll find a balance between the virtual cpu and the memory and the storage and the network performance it's sort of balanced all the components all the features that needs to go in a computer are sort of balanced in general purpose now these instant types are fixed and they cannot be altered because it's hardware based we buy hardware we do not have much control on the hardware that's being used well we have options but we do not have control on the hardware and these instant types are divided into five main families they are compute optimized memory optimized gpu enabled storage optimized and general purpose then as third thing we have to configure the instance now here is where i have a lot of options about purchasing you know what type of purchasing do i want to do do i want to go for a spot instance do i want to go for a reserved instance do i want to go for an on-demand instance these are different billing options available and that's available under configure instance not only that here's where i'm going to put the ec2 instance do i want and public ipads assigned to it do i want an im role attached to it im role is authentication what kind of authentication am i going to provide and the shutdown behavior the shutdown behaviors include do i want to stop the instance when the user shuts down the machine from the desktop or do i want to simply terminate the instance when the user shuts down the instance from the desktop so those things go in here just like the name says configure instance a lot of instance configuration options comes in here that's the third step and not only that under the advanced details or advanced tab under configure instance i can bootstrap the instance with some scripts now bootstrap is nothing but the scripts that you want to be run in the instance before it actually comes online let's say you're provisioning the instance for somebody else you know instead of you launching the instance and then logging in and running some commands and then handing it over to the other person you can create bootstrap shell scripts and you know paste it in a console option available under configure instance and amazon is going to take those commands run it on the instance before it hands over to the user that initially requested for that instance now it could be a different user or just you it sort of automates you know software installation procedures in the instance that we will be launching and not only that there are multiple payment options available under configure instance the user can pick an instance under normal price and that instance would apply normal rates applied to it and there are also options like reserved instance where the user can pay for an instance upfront before a year or before months you know for a span of year or a span of months and that way they can pay less per hour for using that instance not only that you can also go for spot instance like bidding for those instances whoever bids more they get the instance for that particular time well these instances are a lot cheaper than on-demand instances and through bidding and buying you can keep the instance as long as your bid price doesn't exceed the price that amazon is proposing and as the fourth step we will have to add storage to the instance that we are about to launch and here we have bunch of storage options i can go for a permeable storage which is free or i can go for an external elastic block storage also called ebs which is paid and it's a permanent storage or else i can integrate my ec2 instance with s3 for its storage needs and the best part about storage is free subscription users they get to use 30 gigabit of ssd storage or magnetic storage for the whole year in this page where we are ready to add storage we will have to mention or provide the size in gigabit and the volume type is it going to be an uh provisioned volume is it going to be an general purpose volume is it going to be a magnetic volume stuff like that they are volume types and we also need to give inputs about where the disc will be mounted and whether this volume needs to be encrypted or not so all these options are all these inputs are received from us under the adding storage section and then the fifth option would be adding tags like we discussed some time back tags are very helpful to identify a machine in an environment where we have 700 or 1000 vms running and security groups are the actual firewall that sits in front of ec2 instance and it protects that ec2 instance from unintended inbound and outbound traffic now here is where i can fine tune the access to my ec2 instance based on port numbers and based on ip address from which it can be accessed and finally we get to review the whole changes or the whole configurations that we have made to find out whether they are intact with the requirement and then click on submit that's going to launch an ec2 instance but hold on we're not done yet when we're about to launch or before the amazon console actually launches the ec2 instance it's going to give us an option to create a keypad remember i said it's key pair you know key pair is two things one is public and private the private key is downloaded by the user and is kept with the user and the public key is used by amazon to confirm the identity of the user so just go ahead and download the private key and keep it for yourself and this private key gets downloaded as a dot pem file it's a format of the file and it gets downloaded as dot pen file and our next step is to access the ec2 instance and because the instance that we have launched in this example let's assume it's linux instance and that's going to require a tool called putty to be able to access it and this putty tool is really needed when we are trying to access and linux instance from windows instance most of the time windows instance will have putty installed in them but in some rare cases they do not come with putty in those cases we can go ahead and download putty and put the generator and we can start using it to access the linux instance now you might ask well i understand putty watch putty generator now the file that we have downloaded is in dot pem format but unfortunately putty does not accept dot pem format as input you know it has to be converted into a different format called ppk and putty gen is a tool that helps us to convert the dot pem file into ppk file so the quick way to do it is download generator open it up click on conversion and insert the dot pem key that we have downloaded and save the private key and this when we save the private key it gets saved as a dot ppk type private key and when that's done the next very step is to open puti and try to log in and the way to login is to open putty put that ip address here and then click on auth you know this is where we would input the file that we have created so click on auth and then click on browse and find the dot ppk file that we have converted and stored browse it and upload it and then click on open now that's going to open up a login screen for us and the amis comes with a default username depending on the ami that we have picked the username might differ in our case we have picked an ami for which the username is ec2 hyphen user and this is the default by the way let's put the username ec2hyphen user and hit enter and that's going to open up the linux instance using cli there are a few other things that we can do with the terminal that will explain it a little later all right so we have successfully launched an ec2 instance and yeah give yourself a pat on your back launching an instance was just one part of the solution so let's actually talk about how we can notify our customers sns or simple notification service is a service or a product in amazon that helps us to notify customers through email so navigate to sns in your account and create a topic and we're going to use this topic for public notification so let's make it public and then add subscribers to it now these subscribers these subscribers are the people who you want to be notified about the newsletter so we already have the email database in there add them to the subscribers list and then they will start getting new newsletters as in when we post them to the topic and as next step create a bucket in s3 where you can store content and in that bucket create an event that triggers a notification to simple notification service so this is how it'll be set up and notification will be sent to our subscribers anytime we put something in our s3 bucket so s3 bucket is going to create an event for the notification and the notification is going to make sure that it's delivered to your end customer because they're already subscribed to the topic as subscribers and finally let's connect the s3 with ec2 so the bucket and the aws instance are in sync so we put some content in the s3 bucket our email system notifies our customers and the customers can go online to a website that's hosted in ec2 and because s3 and ec2 are in sync the items that we put in s3 will also show up in ec2 see how this is all connected and it's working together and once this is all connected our subscribers will regularly be informed anytime we put new content in the s3 bucket and the same content will be made available in ec2 instance through the website let me welcome you to this new video learning where we're going to learn about lambda one day at office in a growing company there was discussion going on between two it personal it was about the new role that his colleague jessica has taken benjamin the guy standing here wants to know about it and jessica's job is new different and dynamic and interesting too she scale and manage servers and operating systems and apply security patches onto them and monitor all of these at the same time to ensure the best quality for application is given to the users and benjamin was awestruck with the amount of work jessica is doing and with the time it would take to complete all of them but jessica being a very dynamic person very suitable for the job said it was easy for her to handle all of it and even more but that easiness on the job did not last longer as the environment grew more and more it being a startup company jessica was getting drained and was not really happy about all her job as she used to be jessica's manual way of scaling and environment did not last long and she was also finding out that she missed to scale down some of the resources and it's costing her a lot she needs to pay for the service that she was not at all using she sort of felt that she was at the end of the road and there was no way out from this manual task she was very desperate and that's when benjamin suggested something and that brought back the peace and joy jessica initially had benjamin suggested about a service called lambda that can ease the work that jessica is doing at the moment and lambda as happy as it looks like it's a solution that solves the manual repetitive work and lot more and lambda introduced itself as the problem solver and started to explain the following things the very same thing that we're gonna learn about in this series so in this section we're gonna learn about the features of aws lambda and we're going to talk about what lambda is and then we're going to talk about where lambda is being used in the it or in the cloud environment as we speak and then we're going to talk about how lambda works and some use cases of lambda and we're going to be particularly discussing about the use case about automatically backing up the data that's put in the cloud storage let's talk about lambda in detail lambda automatically runs our code without requiring us to provision or manage servers just write the code and upload it to lambda and lambda will take care of it that means that we don't require any server to run or to manage and all you need to do is write the code and upload it to lambda and lambda will take care of it which also means that we can stop worrying about provisioning and managing service the only thing lambda expects from you is a code that's working aws lambda also automatically scales our application by running code in response to each trigger our code runs in parallel and processes each triggers individually scaling precisely with the size of the workload scaling here is done automatically based on the size of the workload lambda can scale the application running the code in response to each and every trigger that it receives billing in lambda is metered on the seconds we only pay for the amount of time that our code is running which means that we're not charged for any of the servers the only payment required is for the amount of time the code is computed with aws lambda we are charged for every 100 milliseconds we are actually charged for 100 milliseconds our code executes and the number of times our code is triggered and we don't pay anything when the code is not running let's talk about what is aws lambda now lambda is one of the servers that falls under the compute section or the compute domain of services that aws provides along with ec2 ebs elastic load balancer lambda is also a service that comes under the bigger umbrella of compute services in aws and lambda allows us to execute code for any type of application with aws lambda we can run code for virtually any type of application or backend services all we need to do is supply our code in one of the languages that aws lambda supports as we speak the languages that are supported by aws lambda are node.js java c-sharp go and python and lambda can be used to run code in response to certain events from other services and based on the event it can run functions and those functions can be of node.js java c-sharp etc now let's talk about where is lambda used there are huge number of use cases for lambda and there are many ways aws lambda is used specifically in the business world let's talk about some of them one use case is aws lambda is used to process images when it is uploaded in an s3 bucket let's say the object gets uploaded in an sd bucket in a format that we don't expect it to be which means the file needs to be formatted so it gets uploaded in a raw format and aws lambda is triggered anytime a new object is added to the bucket and the images are processed and converted into thumbnails based on the devices the other end device that would be reading the data it could be an pc it could be an apple machine it could be an android phone it could be an apple phone it could be a tablet what not so based on the devices different formats lambda can get triggered and convert the video or convert the picture into the different format that it requires another use case for lambda is to analyze the social media data let's say let's say we're collecting the hashtag trending data and the data is received and it's added into the kinesis stream to feed into the amazon environment and lambda action get triggered and it receives the data and then the data is stored into the database which can be used by businesses for later analysis and some of the companies that have gotten tremendous benefit using lambda are thomson routers benchlings not storm coca-cola robot are some of the companies to name at the moment that have received tremendous amount of benefit by using lambda let's look at how lambda works in other words let's look at how the complicated function behind the scenes work in a simple and in a seamless way so here clients send data now clients send data to lambda and clients could be anyone who's sending requests to aws lambda it could be an application or other amazon web services that's sending data to the lambda and lambda receives the request and depending on the size of the data or depending on the amount or volume of the data it runs on the defined number of containers if it is a single request or if it is less request it runs on a single container so the requests are given to the container to handle and the container which contains the code the user has provided to satisfy the query would run and if you're sort of new to containers then let me pause here for a while and explain to you what container is now container image is a lightweight standalone executable package of a piece of software that includes everything needs to run it like the codes the run times the system tools the system libraries and any other settings needed and it is at the moment available both on linux and windows based application and containerized software will always run the same regardless of the environment it's running on and containers isolate software from its surrounding for example there could be difference between a development and staging environment so that's sort of isolated and this helps in reducing the conflict between the teams running different software on the same infrastructure all right now that we know containers needed to understand lambda so if there are a few requests it sends to a single container but as and when the request grows it actually creates multiple containers shares the multiple requests to the different containers there and depending on the volume depending on the size depending on the number of sessions the more number of containers are provisioned so to handle those requests and when the requests reduce the number of containers reduce as well and that helps in saving costs we're not running any resources and paying for it when we're not using them and in fact we're not at all paying for the resources we're only charged for the amount of time a function is running inside these containers now consider a situation where you have to set up a temporary storage and as a system to backup the data as soon as the data is uploaded which is a near real-time backup now a near real time manual backups are nearly impossible and they're not that efficient too and near real-time manual backups that's what the business demands but that's not near real-time backup that to a manual one that's not at all efficient even if we come up with a solution of manually backing up close to near real time that's not going to be efficient looking at the amount of data that will be put in and looking at the random times the data will be put into the source bucket and there is no way we can do a manual backup and they keep it as real as possible but if that's still your requirement we can use aws lambda and set things up so aws lambda manually handles the backup in other words for an impossible or a difficult situation like that aws lambda comes for the rescue and this is what we do create a 2s3 bucket one would be the source bucket where the data will be uploaded and the other one is and destination bucket where the data will be backed up from the source bucket and for these buckets to talk to themselves it's going to require an im role and then for the automatic copy it's going to require an lambda function to copy the files from the source bucket to the destination bucket and what triggers the lambda function the lambda function is triggered every time there's a change in the metadata for the bucket and this data is then uploaded into the destination bucket and after setting all this up we can literally test it by putting a data in the source bucket that's going to automatically replicate or that's going to automatically copy the data from the source bucket to the destination bucket alright let's see how we can replicate data between two buckets well we have a feature to cross region replicate in s3 that's a feature that comes along with s3 what if you want to replicate between two different buckets in two different accounts in those cases we can use lambda to replicate the data between the buckets so you put one data or you put data in the source bucket and that data gets replicated to the destination bucket and let's see how that's done the procedure here would be to have two buckets to begin with and then create an iem role that lets you access to pull data from the source bucket and put data on the destination bucket and then create lambda files or lambda event or triggers to actually look for event in the source bucket and anytime a new data gets added lambda gets triggered copies the data from the source bucket and moves the data to the destination bucket and it uses the iam role and policy for the needed permissions and in just a couple of clicks we have set up a temporary backup system that can run seamlessly without any manual intervention and that can be as near time as possible all right that's the concept and let's see how it is done real time through this lab so to begin with we need two buckets so i have here a source bucket and a destination bucket and the source bucket as of now do not have any data in it so as the destination bucket all right so that's one down the second would be to create an im role right so let me create an im role and the role is going to have this policy in it a policy that's allowing get object on the source bucket and a policy that's allowing put object on the destination bucket and here i'm going to use or i'm going to paste my source and destination buckets arn all right go to services go to s3 source bucket all right click on the source bucket and copy the bucket they are in so that would be the source bucket arn all right on on the destination bucket copy the destination bucket arn and this is going to be my destination bucket arn so with this i'm going to create a policy all right go to im and create a policy now i i have already created a policy with the same information all right destination bucket arn and a policy is available with the name s3 bucket copy lambda attach the policy to the role right go to roles create a role lambda is the service that's going to use it in here attach the policy that we have created right give it a name and then create a role now i have a role created as well all right copy lambda and that's having the policy that we have created sometime back now create a lambda function right so go to lambda create a function give it a name like s3 bucket copy all right choose the role that we want to use right that's the rule that we want to use copy one two create a function all right and in here we're to use node.js code right i have a sample code this can be used as template here replace the source bucket with the name of the source bucket and the destination bucket with the name of the destination bucket this is a node.js code that gets run when an event gets triggered now what's an event anytime there is a new object placed in the s3 bucket it creates an event and the evil triggers lambda and lambda checks this source s3 bucket picks the data puts it in the destination bucket all right paste it here paste it here and click on save right now before you click on save just ensure that you have the appropriate roles defined that's all you got to do click on save all right now i already have created a lambda function all right which is the same thing same code and the role is attached to it now it's running it's active now let's put this to test go to s3 pick the source bucket put some data in it all right and in theory those data should be present in the destination bucket as well there you go it's all done by lambda i'm going to be touring you with different s3 features and technical details in this section so what are we going to learn we're going to learn about what is cloud storage in general and then the types of storage available in general in the cloud and how things were before anybody adopted using s3 that's something we're going to learn and then we're going to immediately dive into what is s3 and then the benefits of using s3 over other storage and then we're going to do a couple of labs or console walkthroughs about what is object and how to add an object what is bucket how to create a bucket stuff like that and then we're going to talk about how amazon s3 generally works now it comes with a lot of features it comes with a lot of promise um how does this all work how does amazon able to keep up with the promise we're going to talk about that and then we will talk about some features add-on features that comes along with the amazon s3 so what is cloud storage in general cloud storage is a service that provides web services where we can store our data and not only that the data can be easily accessed and the data can be easily backed up everything over the internet in charge if you could store your data if you could access the data if you could back your your data everything you do through the internet then that's a good definition for cloud storage and the additional definitions are in cloud storage we only pay for what we use and no commitment no pre-provisioning is you know pay-as-you-go type subscription and the best part is we pay on a monthly basis you know we don't rent a hardware for the year or we don't give commitment for the whole year it's pay as you go and pay on a monthly basis and these cloud storages are very reliable meaning once you put the data in it it's never going to get lost and these cloud storages are scalable assuming i have a requirement to store 100 times of what my actual data sizes and i want it now it's available in the cloud and these storages are secure as well because we're talking about data data virginity they need to be secure and amazon provides tools and technologies through which we can secure our data and these are generally not found in the on-premises storage system so let's talk about the different types of storage in general so s3 is a cloud storage in aws and then we have elastic block store now elastic block store is actually the ssd hard drives that gets attached to our ec2 instance you know it's like the c drive it's like the d drive it's like the e drive that gets attached to our instances now efs is elastic file system the underlying technology is kind of the same but it differs from ebs in a way that ebs can be accessed only by the system that's attached to it meaning the e volumes and the d volumes we spoke about they can be accessed only if there is an instance connected to it but these efs are actually shared file systems elastic file system all right they are shared systems they can be accessed by multiple systems they can be accessed from inside the amazon environment it can be accessed from on-premises equipment as well glacier is actually the archiving solution in the cloud if you want to dump a data and try to keep them in the low cost as possible then glacier is the product we should be using and then we have storage gateway if i want to safely move my data from my local environment to the cloud environment and also want to keep a copy of the data locally so users locally can access them and in a cloud users or internet users can access the data from the cloud if that's your requirement then storage gateway is the one we would be choosing and then we have snowball snowball is really an data import and export system but it is actually an hardware that gets shipped to our premises where we can copy our data into it and then ship it back to amazon and amazon would copy the data into whatever destination we give them in our account and if the data is really huge then i can call for a snowmobile which is actually a data center on a truck where amazon would send me and truck loaded with the data center as you see that has compute capacity lots and lots of storage capacity and electricity ac and lot more so they get come parked near our data center and cables run into our data center we can copy data into it send it back to amazon and they would copy it to our account in whatever storage that we advise them so if the data is really really huge then i would be calling snowmobile snowmobile is not available in all the regions all right let's take this example how things were before s3 was introduced you know two professionals are having a conversation and one of them is finding it very difficult to sort of manage all the data in the organization well if the data is small it can be easily managed but as companies grow and we are living in an era where data is everything we want data to back up every idea we want data to back up every proof of concept that we provide so data is everything so this era it's all about uh collecting data analyzing them and saving you know not losing logs you know saving them analyzing stuff like that so coming back to our discussion here one person finds it very difficult to store and manage all the data that they have so some of the data that this person is having problem uh storing is data's that application used to running and then data that gets sent to the customers and data's that the websites require the data that are because of the email backups and a lot more other storages that an enterprise can have and this person is having problem backing up all those data and even if we think of increasing the local storage capacity that's going to cost the fortune and few things that make it sometimes impossible to increase the storage capacity in-house is you know we will have to go and pay heavy to buy hardware and software to run these storages and we need to hire a team of experts for maintaining them the hardware and the software and anytime if there is a dynamic increase in the storage capacity the on-premises or in-house hardwares won't be able to scale and data security data security is very costly when it comes to building our own storage environment in-house and adding data security on top of it so the other guy in the conversation was sort of quietly listening everything the manager was saying and then he slowly introduced him to s3 because he knew that all the problem that this manager was worried about can be solved to estimate in other words all the scalability all the data security all the not being able to provision hardware and software components are all available with s3 so that actually brings us to the discussion about what s3 is s3 is simple storage service it actually provides an object storage service let me talk to you about object and block storage object storage is where you can store things into drive all right you can't install anything in it and this object storage can be accessed directly from the internet whereas block storage is something that needs to be attached to an instance and we can't directly access it but we can install software in it so that's a high level difference between object storage and block storage and s3 is an object storage what does that mean we can store data from the internet we can retry from the internet but we can't install anything in s3 all right so so s3 is an object based storage and it's it's really built for storing and recovering or retrieving any amount of data or information from anywhere in the internet few other things you need to know about s3 is that this s3 is accessible through the web interface the storage one type of accessing or one way of accessing s3 is by dragging dropping content into it and another way of retrieving data from s3 is go to a browser click on download that's going to let you download any content and the data can be 5 in size now we're talking about one file you know you can have hundreds or thousands of files like that one file can be as big as five terabytes in size and s3 is basically designed for developers where they can push logs into s3 or retrieve logs anytime they want instead of storing locally in the server they can use s3 as code repositories you know where they can save the code and have the applications read the code from there and lot more if they want to safely share the code with another person with a lot of encryption and security added on top of it that's possible as well so there are a few things about s3 and on top of all these s3 provides 11 9 durability and 4 9 availability meaning durability is if i store the data will it get lost amazon is like no it's not going to get lost you're going to have the data because we provide 11.9 durability for the data and availability is if you want the data now will you be able to show it amazon is like yes we will be able to show it we have 99.99 availability and when you request the data we will be able to show the data to you all right so let's talk about the benefits of s3 s3 is durable as we saw it provides 11 9 durability s3 is low cost out of all the storage options in amazon s3 is the cheapest and s3 is very scalable like we were saying that there's no required to pre-provision a storage capacity you know if you need more go ahead and use more if you need even more go ahead and use even more and once you're done some data needs to be removed just remove the data so that particular month you will be paying less so it's very scalable in nature and it's very available as well s3 isn't regional service you know it's not based on one availability zone so one availability zone going down within amazon the whole availability going down it's not going to affect your ability to access s3 storage and s3 is secure a lot of security features like bucket policy a lot of security features like encryption and then mfa authentication are possible with the s3 that actually adds a very good security layer on top of the data that we have stored in s3 and not only that this s3 is very flexible in terms of cost flexible in terms of where i want to store the data in terms of cost there are a lot of pricing tiers within s3 s3 itself is a cheap service now within that we have a lot of pricing tiers depending on the durability so i can always choose to put data on a different storage tier or storage option in s3 we're going to talk about it as you stick along and in terms of flexibility in the region i can always choose any of the region available in the console or in the s3 console to put my data to there is no restrictions on where i can or cannot put the data in the cloud as long as there are regions available for me to move the data to and data transferring with s3 is very simple all i have to do is browse to the bucket upload the data and the data gets uploaded and we can also upload data using cli commands are very similar to linux commands are very similar to what we would use in the run command prompt in windows and what we would use in the run command prompt in the powershell all right let's talk about a basic building block of s3 which is bucket and objects now what's a bucket what's an object object is the actual data and bucket is the folder where the objects get stored let me give you a new definition for objects so object is the actual data plus some information that reference the data like is it a jpeg file the name of the file and at what time it was added to so they're called metadata all right so object is actually data plus metadata and bucket is actually a container that receives the data and safely stores in it and when we add a data in a bucket amazon s3 creates an unique version id and allocates it to the object so we can easily identify it at a later point let me show you a quick lab on s3 all right i'm in the s3 console if you're wondering how i reached here go to services under storage and s3 is right here and let's create a bucket called simply learn now the bucket names will have to be unique so i really doubt if simply learn will be available let's let's check it anyway all right it's doesn't seem to be available so um let me pick another name or let's call it simply learn dot and i'm going to put this in mumbai or i can choose oregon let me choose oregon argon yes let me choose oregon and let me create a bucket sure enough a bucket got created let me upload an object into the bucket and you know these objects can be as big as five terabytes we talked about it right all right let me upload an object all right so that's my object so you get the relation right here is a bucket right here is my bucket and within that is my object now object is the actual file plus what type of file it is and then and then the size of the file the date in which it got added and the storage class it is in at the moment so if i have to access it i can simply access it through the internet so let's talk about how does this s3 bucket work anyway all right so how does it work a user creates a bucket they will specify the region in which the bucket should be deployed we had an option we could have chosen to deploy in all the regions amazon provides s3 service you know beat not virginia beat mumbai beat tokyo beat sydney beat argon and we chose aragon to be the destination region where we want to create bucket and save our data there and when we upload data into the bucket we can specify three types of storage classes in our case we picked the default which was a s3 standard as we saw on the object data it was on s3 standard so that's the basic thing later once the object gets added we can always add a bucket policies you know policies define who access it and who should not access it what can they access are we going to allow users only to read or to read write as well stuff like that so that's bucket policy we're defining the life cycle or the life span of the data in the s3 bucket now over the time do you want the data to automatically move to a different storage tier and at any point do you want to sort of expire the data you know get the data flushed out of your account automatically and those things can be configured in life cycle policies and version control is creating multiple versions if you're going to use s3 for a code repository creating multiple versions let's say if you want to roll back to whatever you had before a month how are you going to do it if you kept updating the file and never took a version of it so version control helps us to keep different versions and it helps us to roll back to the older version anytime there is a need so let's talk about the different storage classes in s3 the different storage classes in s3 begins with s3 standard now this is the default and it's very suitable for use cases where you need less or low latency for example if you want to access the data of students attendance you would retrieve them very quickly as much as possible so that's a good use case to store data in s3 let's understand the different storage classes in amazon s3 now let's take a school for example and uh the the different data is present in a school and the features of those data the validity of the data all right so let's take this example and there are different storage options in s3 let's take s3 standard for example and what would be the actual candidate data that can be stored in s3 standard let's talk about that so s3 standard in general is suitable for use cases where the latency should be very low and in here the good example or the good candidate file that can be stored in s3 is a data that needs to be frequently accessed and that needs to be retrieved quickly something like students attendance report or student attendance sheet which we access daily and then it needs to be retrieved immediately as and when we need it the other type of storage chair in s3 is infrequent access or infrequent data access just like the name says the use case for that is less frequently accessed data i mean the candidate data that should go in infrequent access data is a student's academic record you know which we don't need to access on a daily basis but if there is a requirement we might want to go and look at it and that's going to be quick so it's not a data that we would access on a daily basis but it's data that needs to show up on your screen real quick and the third option is amazon glacier now glacier is really an archival solution in the cloud so for archives high performance is not a requirement so let's talk about some candidate data that can go into archives something like uh students admission fee and it's not critical also now anytime if you want to look at the data you can always wait to retrieve the data so in other words you know put it in the archive and retrieving from the archival takes time so students old record are a good candidate to be put in the archives the other options are one zone ia storage class where the data is infrequently accessed and the data is stored in a single availability zone for example you know by default amazon stores data in multiple availability zones and there is a charge for that now it's not an option but the charge includes storing data in multiple availability zones but if your data requirement is you want to keep the charges in a low even further you can choose a one zone ia storage class where it stores data in one availability zone and the candidate data that can go in one zone in frequent access is students report card and the other option with amazon s3 is standard reduced redundancy storage it is suitable for cases where the data is not critical and the data can be reproduced quickly for example you know take a copy of the library book take a copy of the pdf library book for example now we would have a source pdf and we would make copies of it and then we make it available for the readers to read the other option in s3 is reduced redundancy storage uh here the use case is data that's not critical and data can be reproduced quickly for example books in the library are not that critical and we always have a copy of the book we're talking about pdf so if the customer facing or the student facing book gets deleted i can always copy the same pdf put it in the destination folder and make it available for the users to read that would be a very good use case for reduced redundancy storage all right let's summarize everything that we learned about different uh storage options in s3 so s3 standard it's for frequently accessed data it's the default storage if you don't mention anything the data gets stored in s3 standard it can be used for cloud applications you know content distribution gaming applications big data analytic dynamic websites they are a very good use case for s3 standard frequently accessed data the other one on the contrary is s3 standard infrequently accessed data just like the name says the use case is this is good for data that will be less frequently accessed and and then the use case are it's good for backups it's good for disaster recovery and it's good for lifelong storage of data glacier on the other hand is very suitable for archiving data which is infrequently accessed and the walt lock feature is the security feature of the glacier that also provides a long-term data storage in the cloud this is the lowest storage tier with an s3 the other options are one zone infrequent access storage class just like the name says it's infrequently accessed and it is stored in just one availability zone and use cases are any data that doesn't require any high level of security can be stored here in one zone the fifth storage tier is reduced redundancy storage this is good for data that's frequently accessed it's good for data that is non-critical and that can be reproduced if it gets lost and reduced redundancy storage or rrs is an highly available solution designed for sharing or storing data that can be reproduced quickly all right let's compare and contrast a couple of other features that are available in s3 for example durability availability ssl support first byte latency and life cycle management so in standard the durability is 11 9 durability it's the same for standard standard ia one zone ia glacier except for reduced redundancy the durability is 11.9 and availability of all the storage classes is all the same except for one zone ia where the availability zone is 99.5 percentage all of these products support ssl connection and the first byte latency of these products are most of them provide access with millisecond latency except for glacier it provides a retrieval time of couple of minutes to a maximum of hours and all of them can be used for a life cycle management you know moving data from one storage tier to another storage here that's possible with all of these storage options all right now that we've understood the different types of storage options available in s3 let's talk about some of the features that are available on s3 lifecycle management now lifecycle management is a service that helps us to define a set of rules that can be applied to an object or to the bucket itself life cycle is actually moving the data from one storage tier to another storage chair and finally expiring it and and completing the life cycle of the object with life cycle management we can manage and store our objects in a very cost effective way it has two features basically transition actions and expiration actions let's talk about transition actions with transition action we can choose to move the objects or move the data from one storage class to another storage class with lifecycle management we can configure s3 to move our data from one storage class to another storage class at a defined time interval or at a defined schedule let's talk about transition actions in more detail let's say we have our data in s3 at the moment and we haven't used the data for quite some time and it's that's how it's going to be for the rest of the time so that data is a very good candidate to move to the infrequent access because s3 standard is a bit costlier and s3 in frequent access is a bit cheaper than s3 standard so the kind of usage sort of fits very well for moving that data into infrequent access so using lifecycle transition or lifecycle management i can move the data to s3 infrequent access after 30 days and let's say that the data stayed in infrequent access for 30 more days and then now i realize that nobody is looking into the data so i can find an appropriate storage tier for that particular data again and i can move it to that particular storage chair which is glacier so in this case after 30 days or in a total of 60 days from the time the data got created the data can be moved to glacier and what does this really help us with the lifecycle management help us to automatically migrate our data from one storage cost to another storage cost and by that it really helps us to save the storage cost lifecycle management can also help us with the object expiration meaning deleting the object or flushing it out after a certain amount of time let's say that our complex requirement requires that we keep the data for seven years and we have like thousands and thousands of data like that it would be humanly impossible to check all those or keep track of all the dates and you know when they need to be deleted stuff like that but with life cycle management it is very much possible i can simply create a data and set up lifecycle management for the data to expire after seven years and exactly after seven years the data is going to expire meaning it's going to be deleted automatically from the account all right let me show you a lab on lifecycle management and let me explain to you how it it's actually done so i'm into the bucket that we have created and here's our data into the bucket i assume we have thousands and thousands of data in the bucket and that requires to be put in a different storage tier over time and that requires to be expired after seven years let's say so the way i would create a lifecycle management is go to management from inside the bucket and click on lifecycle and then add and lifecycle rule just give it a name name like expire so all the objects that's present in this bucket meaning the current version of it set a transition for it so the transition is at the moment they are in s3 so here i would like to put them in infrequent access after 30 days and then after it's been an infrequent access for 30 days i would like to move it to glacier all right plus 30 days so how do you read it so for the first 30 days it's going to be in glacier so how do we actually read it after i put the data in s3 the data is going to get moved to standard ia and then it's going to stay in standard ia and after 60 days from the data creation it's going to get moved to glacier so on the 31st day it's going to move to standard ia on the 61st day it's going to move to glacier let's say if i want to sort of delete the data if i want the data to get deleted automatically after seven years you know being in a glacier how do i go about doing it let me open up a quick calculator 365 into seven that's 2 55 days right after that the data is going to get deleted pretty much i have created a lifecycle so after on the 31st day it's going to get moved to infrequent access and on the 61st day glacier and after seven years is over any data that i put in the bucket it's going to get deleted all right let's talk about bucket policies bucket policies are some permission files that we can attach to an bucket that allows or denies access to the bucket based on what's mentioned in the policy so bucket policy is really an im policy where you can allow and deny permission to an s3 resource with bucket policy we can also define security rules that apply to more than one file in a bucket now in this case you know we can create an user or let's say there's already a user called simply learn we can allow or deny that user connection to the s3 bucket are connecting to the s3 bucket using bucket policy and bucket policies are written in json script and let's see how that's done all right there are tools available for us to help us create bucket policies what you're looking at is a tool available online that helps us to create a bucket policy so let's use this tool to create a bucket policy i'm going to create a deny statement i'm going to deny all actions to the s3 bucket and what is the arn to which we want to attach the arn of the resource is actually the name of the bucket but it really expects us to give that key in a different format so the arn is available right here copy bucket arn so this is actually going to deny everybody now we wanted to deny a user just one user now look at that now we have a policy that has been created that's denying access to the bucket and it's denying a user called simply learn pretty much done so i can use the policy and go over to bucket policies so once i save it only the user called simply learn won't have access to the bucket and the rest of them will have access to it so once they save it it gets added and only the user simply learn will not have access to the bucket because we're denying them purposefully the other features of s3 include data protection we can protect our data in s3 with now one of which is bucket policy i can also use encryptions to protect my data i can also use im policy to protect my data so amazon s3 provides a durable storage not only that it also gives as unprotected and scalable infrastructure needed for any of our object storage requirements so here the data is protected by two means one is data encryption and the other one is data versioning data encryption is encrypting the data so others won't get access or even if they get access to the file they won't be able to access the data without the encryption key and versioning is making multiple copies of the data so let's talk about them in detail what's data encryption now data encryption refers to protecting the data while it is being transmitted and protecting the data while it is at rest now data encryption can happen in two ways one is client encryption encryption at rest and server side encryption encryption that's in motion client-side encryption refers to when client sends the data they encrypt the data and send it across to amazon and server side encryption is when the data is being transferred they get encrypted and stay encrypted throughout the transfer versioning is another security feature let i mean it helps us so our unintentional edits are not actually corrupting the data for example let's say you edited the data and now you realize that the data is incorrect and you want to roll back now how do you roll back without worsening it's not possible in other words only with versioning it's possible so versioning it can be utilized to preserve recovery and restore any early versions of every object that we stored in our amazon s3 bucket unintentional erases or overrides of the object can be easily regained if we have versioning enabled and it's possible only if we have one file with the same key name and anytime we update the file it keeps the file name but creates a different version id take this bucket and data for example in a photo.pmg is a file that was initially stored it attached a version id to it and then we edited it let's say we added some watermark uh we you know added some graphic to it and that's now the new version of it when we store it we store it with the same file name it accepts the same file name but creates a new version id and attaches it anytime we want roll back we can always go to the console look at the old versions pick the version id and roll back to the old version id all right let's take this example now i'm in a bucket that we've been using for a while and let me upload an object now before we actually upload an object this bucket needs to be versioning enabled so let me version enable this bucket from this point onwards this bucket is going to accept versioning so let me upload an object photo.jpg let me upload it alright it successfully got uploaded good enough it's uploaded now let me upload another object with the same file name now look at that it was uploaded at 7 40 35 am let me upload another object with the same file name that i have it stored right here that got up to uploaded but with the same name all right so that's the other photo now what if if i want to switch back the way i would switch back is simply switch back to the older version look at that this is the latest version and this is the old version that i have i can simply switch back to the old version that was created at such and such time and i can open it that's going to open the old file so in short it creates different version of the data that i create as long as it's with the same name and at any point i can go and roll back to the original data this is a very good use case if you want to use s3 for storing our codes let's talk about other feature like cross region replication now cross region replication is an very cool feature if you want to automatically keep a copy of the data in a totally different region for you know data durability for any additional data durability or if you want to serve data to your customers who live in another country or who are accessing your data from another country if you want to serve the data with low latency cross region replication is a very cool feature that you can use and get benefited from and let's see how that's done so before we actually do a lab on cross region replication let's put together a proper definition for it cross region replication is a feature that provides automatic copying of every object uploaded to our bucket or your bucket source bucket and it automatically copies the data to the destination bucket which is in a different aws region as you see here in the picture i put data only in region 1 it's going to copy the data to region 2. and for us to use cross region replication versioning must be turned on so it creates versions and copies the versions as well if tomorrow the original region goes down let's say the other region will be active and it has the complete data that was present in the original region or at any point if we want to simply you know cut the region replication and use the other bucket as in standalone bucket it can be used as well because it already has all the data that was present in the master or all the data that was present in the original replication bucket let's see how that's done so there are two things needed one is worsening and another one is roll when we transfer data from one bucket to another bucket we need proper permissions to do so and these roles they give us proper permissions to transfer data from one bucket to another bucket let's see how that's done right so here's a bucket a bucket in usr again let's create another bucket in mumbai call it dot mumbai dot com and put it in uh put it in mumbai or create that bucket in mumbai there you go we have one in oregon we have one in mumbai we're planning to replicate or create application between these two right create replication between these two buckets so go to the first bucket go to management and start a replication add a rule so all content in this bucket is going to get replicated this is my source bucket it's quite simple select the destination bucket now my destination bucket is going to be simply learn.samuel.mumbai.com all right it says well you have versioning enabled in your source bucket but not on your destination bucket do you want to enable it now without which we won't be able to proceed further so let's go ahead and enable versioning through the console that shows up and then like i said it's going to require permission to put data onto the other bucket now i can create different roles these are different roles that are used for different other services i can also choose to create a role that specifically gives permission only to move the data from one bucket to another bucket three months it's done so if i go over to my source bucket and if i add some data to it let's say index.html assuming i'm adding some files to it in theory they should move to the other region automatically they go i'm in the mumbai region and the data's they got moved to the other region automatically let's talk about the other feature called transfer acceleration now it's a very handy and a helpful tool or a service to use if we are transferring data which is very long distance from us meaning from the client to the sd bucket let's say from my local machine which is in india if i transfer the data over to oregon let's say it's a long distance if it is going to go through the internet it's going to go through high latency connections and my transfer might get delayed if it is an uh one gigabit file it's okay but if we're talking about anything that's uh you know five terabyte size and if you're talking about anything that's a five terabyte in size then uh it's not going to be a pleasant experience so in those cases i can use transfer accelerator with which in a secure way but in a fast way or a fastest way transfers my data from the laptop or from client to the s3 bucket and it makes use of a service called cloudfront to transfer or to enable the data acceleration so the way it would do it is instead of copying the data directly to the location instead of copying the data directly to the destination bucket it copies the data locally into a cloudfront location which is available very local to whatever place we are in and from there it copies the data directly to an s3 bucket not going through the internet that helps eliminate a lot of latency that could get added when transferring the data so let's see how that's done right here i'm in the s3 bucket and under properties i can find transfer accelerator and if i enable transfer accelerator so i'm in another bucket let me go to properties and let me go to transfer accelerator and enable transfer accelerator so now if i put data into this bucket they're going to get copied to the local cloud front location and from there they're going to get copied to the s3 bucket from the cloud front now if i need to compare you know how the speed is going to be compared to directly putting the data to the internet and using cloudfront there is a tool available that actually runs for a while and then comes with a report that tells me how much will be the improved speed if i use transfer accelerator and it shows for all the regions available in amazon so from the source to the destination if you want to put you know what's the normal and what's the accelerated speed when you transfer the data those results we will get in the screen so at the moment this tool is going through it testing like uploading some file through the internet and uploading some file using cloudfront and it has come up with the calculation that if i'm uploading file to san francisco compared to uploading through the internet and through cloudfront it's 13 times faster so similar way it's going to calculate for all the regions available and it's going to give me a result welcome to this new learning series called i am a service in aws for security here in this section we're going to talk about what is security in aws and then we're going to slowly or gradually move on to the other topics like types of security available in aws out of all the services why i am is the most preferred one and then we're going to talk about what is im in general how it works the building blocks are components and im and the features it the the attractive features that it provides that makes iam stand out from the rest of the services available we are going to talk about that and then we are going to end today's session with a demo about how iam gets well interacted with the other services in amazon and help create a secure environment in the cloud let's talk about aws security now cloud security is the highest priority in aws and when we host our environment in the cloud we can be rest assured that we are hosting our environment in a data center or in a network architecture that's really built to meet the requirement of the most security sensitive organization and this high level of security is available to us on a pay-as-you-go type meaning there is really no upfront cost that we need to pay and the cost for using the service is lot lot cheaper than what it is in an on-premises environment so aws cloud provides an a secure virtual platform where we can deploy our application now compared to the on premises with aws the security level is very high and the cost involved in using or attaching that type of security to our application is very low compared to on premises there's really no upfront cost and the cost is very lower whatever cost that we pay is very lower than what it would be in on premises and all this contributes to the secure environment we can get through im in aws cloud there are really many types of security services available in aws to name a few are to name the familiar ones im stands first followed by a key management system kms and then we have cognito that's another service and web access firewall vaf is another security service in aws now let's start our discussion by talking about im and that's what this whole series is about now i am or identity and access management it really enables us to manage access to aws services and aws resources in a very secure manner with im we can create groups we can actually allow those users or groups to access some service or we can sort of deny them uh to access the service whichever we want that's all possible through identity and access management and this feature comes with no additional charge now let's talk about how things were before aws before aws or before i am in general it was not that safe in a copyright to share the password over the phone or through the email that was the practice that was existed at that time now we all can remember the days when we need to switch to a different account or we had just one admin password commonly stored in a location or one person would reset it and maintain it and any time or we need to login we call the person ask for the admin password or the phone and then we try logging in now that was not secure at all back then you know somebody could walk by and sort of eavesdrop you know get the password now they walk away with the password with them so they're all possible when we share the password over the phone and through the internet or email now with aws a lot of options are available that i'm not sharing password over the unsecure medium now a slack is a third-party product available with aws it's not an aws product but slack is in third-party application that's hosted on aws and it's really a communication tool that helps people to share documents so now we're not sharing the password over the phone rather through the applications and no eavesdrop person can really catch the password and you know try it in their system to access our environment that's not possible so you see the difference now back then sharing password was only through phone and email and you would write it on a paper and you give it to somebody but now technology is providing provisions enough for us to share the password in a secure way we're still talking about i am im as an web service for securely controlling access to the aws resource now it really helps us to authenticate or sort of limit access to a certain set of users accessing the aws account or certain set of users accessing a certain set of resources in aws account now you see this picture we have an im administrator who is trying to allocate permissions to different group of people so we have group one on the top and group two in the middle and group three uh towards the end the administrator using iam or iam empowers the administrator allow access to certain group to certain resources that's what i am is all about now let's talk about the building blocks or let's talk about the elements that make and complete im so the elements are categorized into six different elements we have the principle in iam the authentication the request the actual request coming in the authorization allowing denying access actions what are the scope of actions can be taken and on what resource is it easy too is it rds is it s3 bucket so what resource these actions are applied to so they are the basic elements of an iem workflow let's talk about principle for example an action on aws resource can only be performed by a principal an individual user they can be a principal a role in aws im it can be a principle as well in other words a principal is an entity that can take an action on an aws resource you know a principle can be an user a principle can be a role a principle can be an application that's trying to access the aws environment secondly authentication now authentication is a process of confirming the identity of the principle trying to access the aws environment now how do we get authenticated or how does a user get authenticated by providing credentials or the required keys to validate that this is what he or she is as per what's in the record so all principle must be authenticated and we get authenticated using the username and password to log into the console if it is cli or we get authenticated using the access key and the secret access key the other component that makes up im is request when a principal tries to use the aws management console what the principal is doing or what the user is doing is sending a request to the aws environment and the request will have the principal actually wants to perform is it a put request or is it a get request or is the delete request stuff like that and it's also going to carry information about on which resource the action needs to be done is it done easy to one is it on s3bucketsimplylearn.com stuff like that so it has a specific action that it wants to perform and the resource on which it wants to perform and also some information about from where the request is originating from is it within an aws environment or is it with another cloud service provider or is it with on-premises stuff like that so these information will be present in the request authorization now this iam uses information from the request context to check for matching policies and it determines it takes a decision whether to allow or to deny the request now here it has a logic by default all the resources are denied and im authorizes your request only if every part of the request is allowed by a matching policy so there is an evaluation logic that says by default all requests are denied and if there is an explicit allow then it overrides the default deny and if you are explicitly denying a service then all the allow statements gets overridden and all the other statements gets overwritten and this explicit deny stands by default only the aws account root user has access to all the resources in that account so if you're not signed in as a root user then we must be specifically granted permission through a policy in other words we need to be authorized through the policy now let's talk about action now after authenticating and authorizing the request aws approves the action that we are about to take now using action we can view the content we can create a content or create a bucket or create an ec2 instance i can edit content or even i can delete a resource that's all based on the action that i'm allowed to do now that was about actions now where is the action performed the action is performed on the resources a set of actions can be performed on a resource and anything that's not mentioned in the action or anything that's not tagged with the action and the resource they don't get to complete for example let's say if you attempt to delete an im role and you also request to access the ec2 instance for that role then the request gets denied now let's talk about some of the other components of im so the other components are im are the basic building block is an user and then many users together they form a group and then policies are the engines that allows or denies a connection you know based on policy one gets to access or gets no access to a resource and then roles roles are another component rules are temporary credentials i would say that can get assumed to an instance as and when needed let's talk about this component called user in an im with im we can securely manage access to aws services and we can create an im user anytime there is a new employee joining our corporate so it's a one-to-one mapping you know one user or one employee get a username directly attached to them and a username or a user is an account specific thing you know it's very local to that aws account that we are using so for in detail an iem user is an entity that we create in an aws environment to represent a person or a service that interacts with aws environment and this user is going to have some credentials attached to them and by default they do not have a password and they do not have any access key or secret access key attached to them in fact literally no credentials of any kind now we must create an user and we must create the type of credential that gets attached to the user do they want to access cli so based on that we would be adding credentials to them a user by default is not authorized to perform any action in aws and the advantage of having one-to-one user is that we can assign permissions individually to that user for example we might want to assign administrative permissions to few users who can administer our aws environment and users are entity within an aws account users don't have to pay separately you know they start using their servers and that get bills under the account they are attached to so separate users doesn't mean that separate payment for those users they are just users in our account and you know the whole account gets to pay for the resources that they provision again the im user does not always represent a person an iem user is really just an identity with associated credential and permissions attached to it it could be an actual person who is a user and it could be an application also a who is a user next we'll talk about groups talking about groups or understanding group is very easy a collection of user forms an iem group and we can use iam group to specify permission for multiple users so that any permission that's applied to the group is also applied to the users who are at add to that group now there are few things you might want to know about group a group can have many users and a user they can belong to many groups but groups can't be nested meaning group can only have users you know a group cannot have a group inside a group that's not possible and there's nothing called a default group that sort of automatically includes all users in the aws account if you want to have a group then we will have to create a group now let's take this diagram for example now this is a diagram of an small company now the company owner creates an admin group for users to create and manage other users as the company expands every day and the admin group creates a developer group and a test group each of these group consists of users users who are a people users who are application that interact with aws environment so the different types of users we have are jim is a user brad is an user and dev app one is a user and and so on so each users have an individual set of security credential like i said it's one to one and the uh 10 users that we see on screen are shared between or they belong to multiple groups in the account and managing group is quite easy we set permission to the group and that permission gets applied to all the users automatically in the group and after we applied the policy if i add another user to the group the new user will automatically inherit all the policies all the permissions that's already mentioned in that group so administrative burden is sort of taken away administrative burden about user a privilege or user permission management is sort of taken out with im all right the next thing we will want to discuss is policy a policy sets permission and controls the access to aws resource and the policies this permissions and controls are stored in json format a policy is a very clear document it defines who has access to a resource and what are all the actions they can perform so in other words policy is an entity in aws that when a policy gets attached to an identity or a resource you know it really defines the permission what they can and cannot do and aws it evaluates these policies when a principal such as a user makes an request and the permission statement and the policy it really defines whether the user is allowed to access a service or is he denied to access a service all right let's take this example now we have a task the task is to give paul who's a developer access to amazon s3 environment and what you're looking at is a snippet of what a policy would look like so these are some questions we will have to answer or these are some questions the policy tries to answer and once it found an answer then it becomes a complete policy so who wants access to the service it's paul who wants access to the service and what action do paul want now paul wants to download or paul wants to upload objects in s3 in json language it's otherwise called paul wants to get information paul wants to put information otherwise paul wants to get object paul wants to put object in s3 and which resource does he want now in this case every bucket the star symbol represents all buckets in x3 so which resource it's all bucket in x3 so we can stop reading already from the data present here so who wants access paul wants access what action does he want he wants to get he wants to put and what's the resource that he's trying to access or you want to give him access it's all the bucket in aws and when does he want these additional statement that goes in a policy so since when does he want that access he wants access till march 2nd 2019 so all this who all this action all this resource you know till the time all this gets allowed so this is the statement that really defines now does this all refer to an allow statement or does this all refer to a deny statement if we had deny here it's the direct opposite of what we discussed but we have allow that means ball is allowed to get and put access in an s3 bucket and all the buckets in s3 till march 2nd 2019 he's allowed to access the resources so if we put together put all the information that we have gathered or all the answers that we have so far if we put together in a json format this is how it's going to look like so the effect is allow or deny here it's allowing and the principle uh is any now who can access it it's any an action it's any bucket in s3 any action in s3 put get request and the resource is any bucket in s3 so that's how a basic policy would look like or that's how a basic policy is formed by answering those basic questions so let's talk about the types of policies so we have two broad categories managed policy and inline policy now manage policy it's the default policy that we attach to multiple entries be users be it groups and beat roles in our aws account on the other hand inline policy are the policies that we create and manage them and these policies are embedded directly into a single user group or a role so if i need to brief it a little bit more then a managed policies are standalone identity based policies that we can attach to multiple users groups and roles in our aws account we can use two types of managed policies in other words managed policies they themselves have two categories uh one is aws managed policies which is created and managed by aws and the other one is customer managed policies just like the name says it's managed by the customer now we can create and edit an im policy in visual editor or simply create a json policy document directly and upload it in the console and start using them so that's manage policy on the other hand inline policies are policies that we create and manage and that are embedded directly into a single user group or role let's talk about this next component called role now this role is very similar to everything that we discussed so far it's a set of permission that defines what actions are allowed and denied by an entity in an aws console the role is very similar to an user i said very similar i didn't say it's the same as the user but i said it's very similar to the user and the difference is that user is hard coded user permissions are hard coded but the role permissions are accessed by any entity you know beat a user or a beat and aws service and moreover the role permissions are temporary on the other hand user permissions are permanent and the role permissions are temporary so in detail a role is very similar to a user in that it's an aws entity with permission policies that determines what the identity can and cannot do in aws now instead of it being uniquely associated with one person a role is intended to be assumable by anyone who needs it roles generally do not have an long term credential you know password or access keys associated with it you know it does not have long term credentials it only has temporary credentials created dynamically and provided to the user as in when the user assumes a role and wants to access any service roles can be assumed by users roles can be assumed by applications roles can be assumed by services that normally you don't have access to aws resources for example you might want to grant a user in your aws account access to your resource that they do not normally have or grant users in one aws account access to resource in another aws account now this is something that we don't have by default and this is something we may not need all the time or you might want to allow a mobile app to use aws resource but you do not want to save the key or save the credential or save the passwords in the mobile app or sometimes you might want to give access to resources who already have identities defined outside of aws like a user who's already and google or facebook authenticator if you want to give them some service or if you want to let them to access some of the resources in your account we can use roles for that purpose or you also might want to grant access to your account to a third party a consultant or an auditor so they can for some time get some temporary access to audit our aws resources remember they're not permanent users just temporary users they want some temporary access to our environmental the audit is over so for those cases roles are a very good use case so let's take this example let's see where role sits and let's see how roles give permission to other services so here is a scenario where where an ec2 instance wants access to an s3 bucket by default though both of them were created by the same username or the same admin they do not have access by default so by default they don't have access and i also do not want to give permanent access you know one-on-one hard-coded permanent access instead i would like to create and share access so that's possible to roll so coming back to the discussion uh the scenario here is that an ac2 instance wants to access data in an s3 bucket so the first thing is to create a role or a permission we saw how that's done right using policies roles use policies policy is the actual working engine for any permission related actions right so create a role with an appropriate policy that gives access to s3's bucket and then launch an ec2 instance with the role attached to it we said rules are assumed by any entity right it could be user it could be an application it could be a service it could be another resource in aws account so here it's another resource in aws account right so we create a role and we attach it to an ec2 instance so now the ec2 instance can assume the role now the same role can be attached to another ec2 instance the same rule can be attached to another user the same rule can be attached to another database service we get the idea right rules can be assumed by anyone so when the ec2 instance wants to communicate with s3 it contacts the role and then the role gives this ec2 instance some temporary username and password that expires after one hour let's say so it's a temporary username and password now with that temporary username and password the ec2 instance can access the s3 bucket and access the file with the credential that it already has and that credential is a temporary one and the credential will expire after one hour or how much ever time the admin sets it to be and then the next time when is when an ec2 instance wants to contact s3 it will have to contact the role and the role will propose a new access key and secret access key in other words a new credential with that new credential the second time the ec2 instance will be able to contact the s3 bucket and get the information so the password is not a permanent or a one-time or a hard-coded password it keeps rotating it's a token it's a secure token that an ec2 instance or any application that uses the role gets to access the resources on the other end well let's talk about the features of iam the main feature of im is that i can create separate username and password for individual users or resources and i can delegate access no those days where we had just one username and password for admin and we need to call the other person to get the username and password over the phone if it gets compromised you know the whole account gets compromised stuff like that those issues are gone i can create separate username and password for all thousand users in my account no more username and password sharing i can manage access to all those thousand accounts in a very simple and an easy way so im provides shared access for all the thousand users in my account it provides share access to the aws environment and the permissions that can be given through im are very granular now we saw how restrictions can be put on get requests you know we can allow the user to download information and not update an information so that's possible through the iem policies that we looked at sometime back and with the use of roles you know we're authenticating an ec2 instance instead of the application running on top of it so if there are like 10 or 20 applications running on top of an ec2 instance instead of creating a separate username and password for all those 20 instances i can create a role and assign that role to the ec2 instance and all the applications running on the ec2 instance get a secure access to the other resources use the role credential that's provided to that ec2 instance now besides username and password im also supports multi-factor authentication where we provide our credential the username and password plus an otp from our phone or an randomly generated number that gets displayed in our phone before amazon console or before amazon cli would authenticate and uh lets us access some of the servers in the account so it's called multi-factor authentication it's username and password plus something else from what you have im also provides identity federation if a user is already authenticated with a corporate ad or facebook or google you know im can be made to trust that authentication method and then allow access based on that authentication now without federation what would happen is the users will have to remember two passwords you know one for on-premises and one for cloud anytime they work on on-premises they'll have to use one set of password and anytime they switch to the cloud which is going to be very frequent in a every day so anytime they switch to the cloud they will have to again type in the cloud username and password now with identity federation we can sort of federate or we can connect both the authentication systems both on premises and im so users can just use one username and password for both on-premises and cloud environments and all that we saw so far it's free to use there is no additional charge for im security there is no additional charge for creating additional users groups or policies its a free service comes along with any account imsn pci dss compliant product now im comes with password policy where we can reset the password remotely or rotate the passwords remotely and we can set rules like how a user should pick an password it should not be the one that was used in the past any of the past three times stuff like that so uh you get the idea right how the features of iam help us to build and a secure authentication and authorization system in or for our aws account all right it's time to do a quick lab let's let's put all the knowledge that we have earned so far together and apply it and try a lab and solve the problem so here's a problem statement we need to create an s3 bucket for a company where users can read or write the data only if they are multi-factor authenticated so there could be like 20 users in an account but a user is allowed to read or write a data in an sd bucket only if the user is multi-factor authenticated you know this is a very good use case if we have sensitive data in an s3 bucket and you only want privileged or mfa authenticated users to do changes to those buckets and for those privileged users you would enable multi-factor authentication so let's see how that's done before that let me talk to you a bit more about multi-factor authentication a multi-factor authentication is that additional level of security process it's provided by aws here the user's identity is confirmed only after the user proposes or the user passes to level of verification now one being the username and password and another being an otp that gets generated in this case it's the mobile phone it could also be an rsa token key that generates the one-time password that's possible as well but in this case it's the mobile phone now that's going to be very similar to how we log in to our gmail account sometimes when we log into a gmail account and i mean if you have the proper setting enabled it's going to send an one-time password to mobile phone and only after we put that information it's going to let us log in let's see how all this is done in an aws account so the first thing is to log in to the aws account create a user and attach the user to the virtual mfa device now we're going to use google authenticator here and we're going to attach the user to google authenticator and make the user use the google authenticator one time password every time he logs in to the account now this is an addition to the username and password that he or she already has right so the first thing is to log into the account connect or sync the virtual appliance with the aws username and password now when we sync there will be an one-time password that gets shown in the phone type in the password in the aws console and then the phone and the username comes to sync and from that point onwards the user anytime they log in they will have to propose their username and password which is the first step of security and then once they typed in the username and password type in the mfa code that gets generated on the phone that would be the second step or security or the last step of security once that is done the account is going to let them log in and access the aws resources alright so in order to test im mfa s3 together this is what we are going to do so we are going to create a bucket and we're gonna have two users and we're gonna allow or deny access to the s3 bucket to those two users based on whether they are mfa authenticated or not and on purpose uh we're gonna assign mfa to one user and no mfa to the other user like you might have guessed by now the user with mfa will have access to the bucket and the user with no mfa attached will not have access to the bucket both of them are going to have full s3 services but mfa like i said is one layer above the other permissions mfa stands at all time so irrespective of whether the underlying policy says but they have full access to the s3 bucket they're not going to because mfa stands on top of it let's see how that's done to begin with firstly we need a bucket so let's create a bucket so i'm in my aws console so let me go to s3 and in here create a bucket call that bucket as simply learn mfa bucket and let me put it in north virginia and create all right so that's done now i'm going to put a separate folder in here i'm going to create another folder in here and the name of the folder is tax documents now this is where i'm planning to keep my tax files and i only want my privileged users to be able to upload and download information from the tax document so on the bucket side it's partially done or on the sd side it's partially done so let's go back to iam and here we're going to create two users one a junior employee and another one and a senior employee the senior employee is the one who is going to have mfa access and more access to the s3 bucket and the junior employee is the one who's not going to have access to that particular bucket so let's get started let's create an user under iam click on users and let's create a user called as user junior right so he's a junior user and for the password set a password the user does not need to create a new password at sign in because it's a test environment and the user who creates the username and the user are the same person so let's keep it simple username password management console access and this user is going to have full s3 bucket access all right this is really the policy and we should be familiar with this json document by now it says effect is allowed and what action any action in s3 and what resource anybody who is attached to it review and create a user so this user is having s3 full access now let's create another user call them as senior employee all right attach a policy to the user you know from the surface level both of them have the same permissions both of them have s3 full access all right so both of them have s3 full access as we see there's one thing different we're going to do to this senior employee now we're going to attach mfa to this employee so the way we would do it is under the summary section of that particular user click on security credential and see here is a place we can attach an mfa to this user so let's attach mfa to this user for mfa i can use both virtual mfa or hardware mfa this being in lab environment we're going to use virtual mfa now that is an application there are a lot of applications available but there is one application that i'm using at the moment called google authenticator so once we have installed the application now once it is running so i can go to virtual mfa device and once it is running i can scan the code using the google authenticator in my mobile phone once you turn on the application and once you do scan the barcode that is showing right here now that's going to give us codes that we can use and we will have to put in those codes here twice we'll have to wait for some time and it would generate a second set of key that we can put so my first set of key is four zero two three five one right i'll have to wait for some time and then it would throw me a second set of key it's like validating twice right here so let me put in my second set of key so this user is now mfa authenticated user so anytime this user logs in it's going to ask for mfa anytime this user proposes something it's going to attach the mfa key along with it all right so we're done with iam what have we done with im we have created two users both of them have actually full access but one of them have mfa access and the other one does not have mfa access and based on whether the user is having mfa or not you know we're identifying privileged users by this mfa all right so based on whether the user has mfa or not are we going to allow deny access in the s3 bucket so we already have a bucket we already have a folder and this is a very privileged folder part here i would like to restrict access based on whether the user is having mfa or not so let me create a bucket policy here now here's a policy that i have written the policy says that you know under tax document folder give access only if the user is mfa authenticated right if the user is not mfa authenticated simply deny the access to that user so as you might guess right now the user one will not have access or the junior employee will not have access because it's a privilege then you know junior employee might tend to delete them unknowingly and only privileged users get access to the tax document because it's confidential because it's privileged because if it gets deleted we won't be able to get it back for those reasons right so apply the bucket policy now try logging in as the different users and see how the permissions are applied on both the users all right so let's make a note of the im url and then the two usernames and password let's login so let me login as the privileged user sure enough i'm being asked for an mfa code let me put my mfa code so let's directly navigate to the s3 bucket right here's my privileged path let me try uploading some content claim forms sure enough i'm able to upload content into it will i be able to delete yes i'm able to delete as well now let's see what's the situation for the other user so let's login as the other user navigate to the sd bucket if you noticed when logging in it did not ask for an mfa code because the user is not set up for mfa so navigate to the folder which we want to test now try adding some files as you see it's failing to add content to this bucket it shows error because the bucket policy is requiring mfa and this user is not mfa and we did that in iam or under iam but still as you see you know they will be able to view the content in the bucket they can do all that we have specifically restricted access to upload and download content from the bucket and that's exactly this user is now denied with welcome to this new video lesson where we're going to learn about plot formation in this section we're going to learn about why we need a solution called cloud formation or why we need a product called cloud formation and we're going to talk in detail about what is cloud formation and then how cloud formation works and the concepts involved in cloud formation like you know templates and stacks there are some concepts involved in cloud formation we're going to talk about that and how do we maintain access control over the cloud formation we're going to talk about that as well in this video and then we're going to go through a demo or a lab where we create an lamp stack environment from a template through cloud formation and then we're going to talk about some use cases one of which include being able to create an redeployable template using cloud formation that's a very good use case we're going to talk about that in this video so let's talk about why cloud formation so when the number of products and services grow more and managing them and maintaining them becomes and tedious work and cloud formation it sort of eases that environment now cloud formation is really an infrastructure as a code and it sort of solves all the problem that we discussed some time back and with cloud formation we can actually create a perfect clone of the server configuration at any point in time and we can also manage the configuration changes and configuration changes with cloud formation we can manage the configuration changes across the environment and not only that we can easily embed in the ad hoc changes to our existing environment so one might ask why use cloud formation let's look at the current problem that we have creating and managing multiple aws resource in the portal is a big task especially when you need to replace some services it becomes a bigger task now cloud formation eases that work let me explain it in detail now at the moment without the cloud formation or environments that don't use cloud formation have this problem that is a lot of time is being spent on building the infrastructure and less focus and less time at the moment is being given on developing that application that runs on that environment that's because majority of the time is consumed by building the environment and if we need to redeploy the same environment again and again the same happens in a cycle again in a new environment we start from the scratch build environment and then put application on top of it now that can be avoided using cloud formation now without cloud formation still is a problem because the the resource or the bandwidth that is needed for the application development is not provided because that's taken by developing the infrastructure and to solve that problem we can use cloud formation now that leads to a discussion about what is cloud formation so let's talk about or let's try understanding what cloud formation is cloud formation is a service that provides users a simple way to create and manage collection of aws resources by provisioning and updating them in a very orderly and in a predictable way if i need to expand explanation of cloud formation cloud formation is an service that really helps us to model and set up our amazon web services or resources so we can spend less time on managing the resources and show more focus on the application that runs on top of it we can simply create a template for an ec2 instance or for an rds instance and upload it in the cloud formation portal and cloud formation will provision and configure those resources based on the template that we define we really don't have to sort out the complexities in dependency between those applications once the template is vetted and validated cloud formation takes the responsibility of handling the dependencies between the application in an orderly and a predictable way it creates those services and makes it available for us once the template is fully run so in short at the cloud formation it allows us to create and model our infrastructure and applications without we having to perform the manual action for example the well-known company called expedia is able to easily manage and run its entire front and back end resources they run on aws in cloudfront what does that mean they are spending less time on managing the infrastructure and more time on the core business and the application that runs on top of it and we're still talking about what cloud formation is with cloud formation it enables us to manage our complete infrastructure or aws resource in a text file now we're going to see that in a moment you know it's either a json or an yml file and that file is called as the template to begin with the resources the template provision is called the stack all right so the abstract of the code is called the template and the resource that it provisions is called the stack and this stack can be i mean it's going to run resources obviously and those resources can be updated as well it's not like once you create a stack gets done if you need a change you need to go create another resource that's not needed this stack is an updatable stack let's say if i'm including two more servers if i'm sort of extending my environment it's branching out to another application now i'm including another functionality in my environment all those can be embedded in the update and can be updated so like i said the stack is an updatable one now let's talk about what the template can do or some features and functionality of the template now using a template we can almost include all the application and resources that a user might require let's say you know after the easy to instance is provision running an application on top of it and you want to templatize that and include that in the template that's possible as well and these templates are portable ones meaning i can use the template in one region and i can use the same template in another region and that's going to build a very similar environment in another region i can also share that with the customer and when they run it it's going to run an environment from the portable template in their account in their environment of course all the security is taken care i'm just saying that if we need to build a resource if you need to build an architecture if you need to build an environment which looks very similar to the other environment that is possible and and these templates are usable by the use of some sections in the template like the parameter section the mapping section the condition section within the template is what makes the templates reusable and we will talk about parameters mapping and conditions down the line we're still talking about aws how aws cloud formation works and we're going to talk about how a template becomes an infrastructure right so to begin with one requires a template and the template would be in json or yml format so one would develop a template based on the requirement based on number of ec2 instances based on whether they want a load balancer based on what they want and a windows server based on whether they want an database server in that environment and then based on the applications that will be running on top of it that can be templatized in the code itself so one creates a code and then the code is saved locally or they can be put in an s3 bucket from which the cloud formation will call the template and provision the resource right so we use cloud formation uh to create a stack or create the resource defined in the template and cloud formation analyzes the template first validates template and then it identifies the uh dependencies between them and then it starts provisioning the resource one after the other for example if you need a five servers in a new vpc obviously it's going to create the vpc first and servers second and stack all right let's understand the two major components in cloud formation by now you might have guessed it the two major components are the template and the stack together the aid or the complement cloud formation and that's how the resource of provision that's how the resource are managed and that eases the job of the administrator who's managing the environment so let's talk about template first a template in cloud formation is an text file or a formatted text file in json or yml language and that the code or the template or the text file that actually describes how your infrastructure looks like or will look like once provision and for us to create the template or for us to modify an existing template we can use a tool available called cloud formation designer from the console or any text editing tool that's available but i would recommend the cloud formation designer though that has rich graphics in it and these templates are built off or they consist of nine main objects the first of which is the format version and this really identifies the capabilities of the template based on the version number and then the second object in a template would be description and this helps us to include any arbitrary comments about the template for what reason are we building this template what's the purpose of it stuff like that and then we have the object called metadata in a template and that includes a details about the template details of the resources in the template and then we have the optional parameter section this really helps us to customize our template now parameters enable us to input custom values to our template each time we create or update and stack and the mapping object it helps us to match a key to a corresponding set of named values for example if you want to set a value based on region then we get to create a mapping that uses region name as the key and then the value that we want to specify for each specific regions and then we have conditions object this is again an optional object with this we can include statements that define when a resource is created or when a property is defined for example we can compare whether a value is equal to another value and based on the result of that condition we can conditionally create additional resources and then the object called transform it's it's an optional object as well and this transform section specifies one or more transforms that we can use in our cloud formation template for example uh aws cloud formation transforms helps simplify template authoring by condensing the expression of the aws infrastructure as a code or the cloud formation template and it enables reuse of template components for example we can condense multiple line resources declaring it into a single line in our template now we will talk about all these examples in the upcoming slides and then we have the required section called resource that declares the aws resource that you want to include in the stack for example if you want to include an ec2 instance if you want to include a nested bucket we would be using the resource object in our stack and then we have object called output this output object it's an optional object in the template and it helps us to sort of declare the outputs that we can import into other stacks or we can actually show it on the cloud formation console for example if you want the output of the or the name of the s3 bucket that got created in response to a cloud formation template now we can have that outputted into an output section in the cloud formation console itself so it's sort of easy for us to identify the resources that the template provisioned so with all of those objects put together this is how a full-blown clot formation template with all the objects included is going to look like now this includes optional and mandatory object let's talk about those objects in detail in the upcoming section all right let's now discuss each and every object of this template structure and let's begin our discussion with the format version the format version it really defines the capabilities of the template the latest version format version as of now is the one that was updated on 2010. now the value of the template format version it has to be and string and if we really don't uh specify the format version cloudformation automatically assumes the latest version talking about description descriptions are really any comments that help ourselves that we want to ember in the template itself that can be a description and this description in a template it has to be an string length of the string can be between 0 to 124 bytes in length the metadatas are really used in the template to provide any additional information using the json and yaml objects and metadata holds some implementation details of a specific resource and parameters are really a way to customize the template each time we create or update our stack these parameters are the ones that helps us to give some custom value to the template during runtime in the example shown below the parameter named instance type parameter it lets us to specify the ec2 instance type for the stacks to use when creating or updating the stack by default as we see it's picking t2 micro and this is the value that will get picked if we don't provision another value to it and mapping mapping really enables us to map a key to a corresponding named value that we specify in a conditional parameter and also we can detrive the map or to try the values in the map using the function find in map instrict function let's take this example let's say based on a region if you want to set values in a template we can create a mapping that uses a key and holds the value that you want to specify for each region and the object conditions can be used when we want to reuse the templates by creating resources in a totally different context for example in a template during stack creation all the conditions that we mentioned in the template are evaluated and any resources that are associated and that come up to be true are created and any condition that fails or invalid are kind of ignored automatically and the check moves forward to the next condition and conditional is an optional object and it includes statement that define when a resource is created or when a property is actually defined for example you can create whether for example you can compare whether a value is equal to another value and based on the result of that condition we can conditionally create resources if you have multiple conditions then they are generally separated with commands sometimes conditions are very helpful when we want to create templates and we want to reuse those templates which was created for one environment to another environment for example if you want to use a template that was created in a test environment in a dev environment conditions would be helpful for example in a production environment we might want to include instances with certain uh capabilities and in a test environment we might want to include instances with some reduced capability in them for saving money and condition helps us to sort of create resources based on the environment it will be used in we can also use intrinsic functions to define conditions like equals we can also use r conditions we can use not functions intrinsic functions in a condition and when put together the syntax is going to look like the one that you see on the screen now the optional object called transform it specifies one or more transforms that the cloud formation uses to process our template now these transform sections are built on a simple declarative language of aws cloud formation with a powerful macro system that helps us to reuse the templates or the template components and they really help us to condense or simplify our codes by enabling reuse of the template components for example condense a multiple line resource declaration into a single line in our template the resource section it really declares the aws resource that you want to include in the stack such as ec2 instance such as s3 bucket and some of the resource fields are the logical id the resource type and the properties let's talk about them in brief the logical id is unique within the template and the logical id name it actually reference the resource in other parts of the template in this case it's my bucket for example i know if we want to map an elastic block store volume or if you want to map and uh name of the bucket uh in another place in the uh template now then we would use the logical id and again in this case it is my bucket a logical name has the physical id uh with which we actually assign the resource for example in this case my s3 bucket was named as my bucket to begin with and the resource type in the template it identifies that the type of the resource is an s3 bucket and the properties are the additional optional information that we can specify to our resource for example if it is an ec2 instance let's say we would like to know which ami it used then property section really defines the ami that was used or ami as the property of the instance so this section describes the output value that needs to be imported to other stacks or the output value that needs to be shown in the console for the user to easily navigate to the resource that are being created for example for a s3 bucket name we can declare an output and once the bucket is created we can actually declare the output and sort of uh import the name of the bucket in a different place in the console so it's it's easy to view instead of you know the user getting confused and going to s3 and searching for the bucket name this can be made available in the cloud formation console itself and the output fields it can include some of these followings like the logical id uh it's actually the identifier of the current output and it's very unique within the template and it can also include a description description is really a string type that describes the output value and it can also include value and value is the property that was returned by the aws cloud formation describe stacks command all right let's talk about template resource attribute in this section we're going to talk about attributes that you can add to a resource to control additional behavior and relationships so let's talk there are many resource attributes available out of which we have i mean if we need to name them we have a creation policy and then we have deletion policy it depends on metadata attribute update policy attribute let's talk about each of them separately when we associate the create policy attribute with the resource it actually delays the resource configuration actions before proceeding with the stack creation i mean the attribute stack creation is delayed till the time the cloud formation receives some number of success signals back i mean this is a very good use case when we do auto scaling and this creation policy can be used only with a limited number of services like auto scaling and ec2 instances some of the use cases would be if you want to install and software application on an ec2 instance you might want the applications to be running before it proceeds further so in this case we would use the creation policy attribute to the instance and then send the success signal after the applications are installed and configured then cloud formation goes to the next step here in the syntax that we're looking at the auto scaling creation policy and the minimum successful instance percentage it's actually a number of signals that it will have to reply back before it can move forward using deletion policy it preserves backing up of resources when the stack is getting deleted and by default if you don't have a delete policy the data gets deleted just like that but if we use an deletion policy we would be able to preserve the data by taking a snapshot of it or just letting the resource and moving on with deleting the other resources in the stack and we specify the delete policy attribute for each resource specifically that we want to control and if there are no deletion policy mentioned for a particular resource the cloud formation simply deletes their resources as that's its default behavior and if you want to keep the resource when the stack is deleted we use we mention retain in the delete policy and that simply retains the resource from getting deleted so in this example we're actually retaining a dynamodb table rest of the resources in the stack will get deleted and using that depends on attribute we actually create a an order in which the resource gets deleted for example if i map one resource to another resource and say that this resource depends on the other resource the other resource gets created first then followed by this instance or this resource so with the depends on attribute we can specify that the creation of specific resource is actually followed by an another when we add the depends on attribute to a resource the resource is created only after the creation of the resource specified by the depends on attribute let's take this example in this example we're saying that the resource x depends on why we've said that x depends on y so in this case when cloud formation is executing it's going to deploy y first because it has some resources that depends on it so it's going to create y first and then it's going to create the resource called x on the same context let's say in the template we say that an ec2 is going to depend on s3 and when the cloud formation is executing it's going to create an s3 bucket first and then the ec2 instant that's probably because the codes needed for the ec2 instance are stored in the s3 bucket and and what we're looking at is the syntax of how the depends on template resource attribute will look like this is how it actually will look like when we put it in the cloud formation the instance depends on the bucket the other template resource attribute is metadata metadatas are the one that really helps us to associate our resources with the the structured data and by including this attribute in our template or by including this attribute to a resource we can specify the data in json or in yaml language for example in this section of the template we're creating a vpc and the metadata explains our metadata defines that the vpc is part of the region us east one and the update policy template resource attribute and cloud formation we can manage and replace the updates of the instance in an auto scaling group for example you're looking at this sample where it says that the update policy will it replace the instance in an auto scaling group or not so depending on what the value is for the will replace section whether it's true or false depending on that the update policy will replace the instance or will replace the auto scaling group itself when it's doing and rolling deployment now that we've discussed about templates let's talk about aws stacks as well now we know that aws stacks are the actual resources that yes generated or that gets created because we ran a template right so stack is a collection of aws resource that we can manage in a single unit in other words you know we can create a stack update a stack or delete a stack and that would actually create update delete the collection of resources within the stack and all the resources in the stack are really defined by the cloud formation template and a stack for instance can include all the resources required to run a web application such as a web server a database server and some networking rules that come along with it all that gets embedded in a stack and let's say if we want to delete all the resources that were created by template that is in a separate stack we can simply delete the stack and that would sort of delete all the resources that is in the stack so it sort of helps us to manage all the resources in a single unit or as a single unit so to summarize stack is a collection of aws resource and it can be managed as a single unit and the cloud formations template defines a stack in which the resource can be created deleted or updated in a very predictable way and a stack can have all the resources like web server database server vpc the network and the security configurations in a vpc all that is required to run a web or a database application now stack could be nested as well with cloud formation we can actually nest a stack with another stack and that's going to create an hierarchy of the stacks for example i mean this is a very good picture that explains how the stacks are nested in a hierarchical way let's say stack p is the root stack and that could be a parent stack for the stack called r and the stack call q and the same way the stack call r is a parent stack for the stack s and the stack s is the parent stack for d and q you get that right so stacks can always be nested it's possible cloudformation allows us to create a microsoft windows stack based on amazon ec2 windows amis and provide us with the ability to install software and use remote desktop to access our stack and update the configuration of our stack and these stacks can be run on windows server instances there are a number of pre-configured templates available and it's directly available to use from the aws website itself to name a few we have templates available for sharepoint running on an uh windows server 2008 r2 we also have templates that create a single server installation of active directory running on a windows server and in fact we have templates available for elastic bean start and some sample applications that runs from an windows server 2008 r2 let's talk about stack sets stacks sets actually has taken the stack level implementation to the next level using the aws cloud formation template we can define a stack set that lets us to create stack in aws account across the globe by using just a single template and after we create the stack or after the stack set is defined creating updating or deleting the stack in the target account and the regions can also be specified by us so the stack said it really extends the functionality of the stacks by allowing us to create update and delete stack across multiple accounts regions in just a single operation and using an administrator account we can define and manage an aws cloud formation template and use the template as the basis for provisioning stacks into you know some selected target accounts across a specified region let's talk about cloud formation access control with im i can specify the list of users or list of privileged users who would have access to the cloud formation template who can create update and delete stack so that's how im helps me with having or putting a layer of control over the cloud formation service and we can also use service roles service roles are the one that allows aws cloud formation to make a call to a resource in a stack on the user's behalf and i can also have stack policies and these stack policies are applied to the users who already have access to the cloud formation service and these policies are the ones that help the users who attempt to update the stack all right let's do a quick lab on creating and lamp stack on an ec2 instance with a local mysql database for storage now we can do the other way as well i can pick an ec2 instance and install the servers one top of the other that can be done as well but that's going to take a lot of time now we're talking about automating it we're talking about you know provisioning and infrastructure using a code and cloud formation helps us to do that so this is what we're going to do we're going to create or we have an cloudformation template already and we're going to use that template the provision web instance and that is going to have look at that that's going to have my sql installed in it that's going to have php that's going to have httpd servers installed in it and running and it's also going to have an a php website running from it so that's going to receive information from the website and that's going to store that information in the database that's very local to it this is the template that defines that environment all right this is a very simple provisioning so let's provision this environment let's create a stack we're going to give a stack name and a database password right and which region is it this is in california is it all right now let me do one thing let me switch to the mumbai region or let me switch to north virginia region not virginian from here actually let me switch to mumbai region all right just to be sure it's the same template that we're provisioning right that's the same template that we are provisioning all right let's give this a name so it can be great this could be simply learn simply learn lamp stack and what could be the db password and db root password and what could be the db user i'm going to keep the rest as defaults and i'm going to launch all right so there's an issue with the password [Music] password should only have alpha numeric characters get that so let's go back and create a new password it also needs to have eight characters so what can be your password look at that the template is actually creating the resources and here in the screen we can see the template that we used to deploy the environment if you remember we have we discussed about instant types in parameters right the default instance type is t2 small but these are also allowed to run all right parameter section is the one that was used to receive information from us like the db name the db password the db root password the user instance type key name and ssh locations all right now this resource section is is showing us the resource that it actually created right looks like it's rolling back why would that be give me a moment [Music] all right this is one live example anytime it could not provision or any if there is an interruption that it could not proceed it would simply roll back and delete all the resources that it had provisioned and that's what it did now let me fix this all right as a fix i have switched the regen because i had some issue with the vpc that's going to take time for me to analyze and fix it so here we're going to understand about cloud formation so i thought i'll run the cloud formation from another vpc that i know is working fine that's in not virginia all right let's provision this lamp server which is running out of linux which has apache installed in it which has php and mysql already installed in it so one server gonna act as web and db server all right so i'm going to create a stack and from here i'm going to pick a sample stack if we look at if we look at it in the designer it's going to provision one lamp server and a security attack security group attached to it there you go right so let's go ahead and create a stack all right parameters are used to receive input from us what could be the db password and the db user and let's select the key name right i'm going to keep the rest as defaults and i'm going to create a stack all right so our stack that we run from the template at the moment it's getting created it's in progress from here i can watch the events as to what is getting created at the moment like i said in a lot of these would have already sorted out their dependencies you know which one needs to get created first which one needs to get created second all that is sorted out by cloud formation itself and parameters are the values that we have inputted and a resource tab is a section where we can find what resources are being provisioned at the moment so to begin with it created a security group at the moment it's creating an instance and output it's empty as of now we're going to come back and see it now output is where we can get outputs of a finished job let's say here i wanted an url of the server that got created so instead of me you know going to a couple of other places within the console and find the output of the url or the the output of the resource that was created i can simply get it from here well this needs to be defined in the templates first once we define it in the template it's going to be made available like here right so i can open it up and there you go so it has it already has a database locally installed and it's connected to where if i launch an application on top of it that will get connected to the database which is very local and my single server with web and database in it is ready for production now while we're talking about a lamp server a single lamp server let's talk about uh creating an environment like this which has elastic load balancer a vpc a server running in an auto scaling group and putting some restrictions that nobody will be able to access the ec2 instance directly but they should be able to access the ec2 instance or the data only through the elastic load balance so this is a bit complicated than the environment that we've already created let's see how this is done all right in this lab we're going to create elastic load balanced auto scaling group which is going to create a load balancer and an auto scaling group and that auto scaling group receives traffic only from the load balancer so here's the template and here's the architecture that we're going to provision let me call it right and the key pair it's going to use is simply learn key pair now this is where the problem was now we were selecting few subnets so now if i provision it now this is going to take its short time to provision the resources so through parameters we can see the values that we have inputted and the events look good no rollback no no deletes and the resource section shows the resource that it is creating so previously created a target group now it's creating the application load balancer itself all right so it created an um load balancer and then it created a security group it created a launch configuration it at the moment it's creating a web server group and the events look good with all green and yellow green ones are completed yellow ones are in progress we don't have any red delete or roll back errors so it looks good all right it took a while and the resources have been created some of them show that they are in progress but the status of the stack is complete all resources have been created and we do have an output to access the url there you go and remember there was another feature of the cloud formation that was it should restrict access i mean direct access to ec2 instance let's see if that has been done if i try to access using the load balancer it lets me access but if i try to access it using the ec2 instance ip or the dns name my connection won't go through because it's blocked through the security group but direct access through the load balancer is allowed and that's what we wanted to achieve through that template here we're going to talk about amazon ecs a service that's used to manage docker containers so without any further ado let's get started in this session we would like to talk about some basics about aws and then we're going to immediately dive into why amazon ecs and what is amazon ecs in general and then it uses a service called docker so we're going to understand what docker is and there are competitive services available for ecs i mean you could ecs is not the on and only service to manage docker containers but why ecs advantage of ecs we will talk about that and the architecture of ezs so how it functions what are the components present in it and what are the functions that it does i mean each and every component what are all the functions that it does all those things will be discussed in the architecture of amazon acs and how it works how it all connects together that's something we will discuss and what are the companies that are using ecs what were the challenge and how ecs helped to fix the challenge that's something we will discuss and finally we have a wonderful lab that talks about how to deploy docker containers on an amazon ecs so let's talk about what is uh aws amazon web service in short called as aws is an web service in the cloud that provides a variety of services such as compute power database storage content delivery and a lot of other resources so you can scale your business and grow not focus more on your it needs and the rest of the id demands rather you can focus on your business and let amazon scale your it or let amazon take care of your it so what is that you can do with aws with aws we can create deploy any application in the cloud so it's not just deploying you can also create your application in the cloud it has all the tools and services required the tools and services that you would have installed in your laptop or you would have installed in your on-premises desktop machine for your development environment you know the same thing can be installed and used from the cloud so you can use cloud for creating and not only that you can use the same cloud for deploying and making your application available for your end user the end user could be internal internal users the end user could be the could be in the internet the end user could be kind of spread all around the world it doesn't matter so it can be used uh to create and deploy your applications in the cloud and like you might have guessed now it provides service over the internet that's how your users worldwide would be able to use the service that you create and deploy right so it provides service over the internet so that's for the end customer and how will you access those services that's again through the internet it's like the extension of your data center in the internet so it provides all the services in the internet it provides compute service through the internet so in other words you access them through the internet it provides database service through the internet over the internet in other words you can securely access your database through the internet and lot more and the best part is this is a pay as you go or pay only for what you use there is no long term or you know beforehand commitment here most of the services does not have any commitment so there is no long-term and beforehand commitment you only pay exactly for what you use there's no overage there's no overpaying right there's no buying in advance right you only pay for what you use let's talk about what ecs is so before ecs before containers right ecs is a service that manages docker containers right it's not a product or it's not a feature all by itself it's a service that's dependent on docker container so before docker containers all the applications were running on vm or on a host or on a physical machine right and that's memory bound that's latency bound the server might have issues on and on right so let's say this is alice and she's trying to access her application which is running somewhere in her on premises and the application isn't working what could be the reason some of the reasons could be memory full the server is currently down at the moment we don't have another physical server to launch the application a lot of other reasons so a lot of reasons why the application wouldn't be working in on premises some of them are memory full issue and server down issue very less a high availability or in fact single point of failure and no high availability if i if i need to tell it correctly with easiest the services can kind of breed free right the services can run seamlessly now how how is that possible those thing we will discuss in the upcoming sessions so because of containers and ecs managing containers the applications can run in a high available mode they can run in a high available mode meaning if something goes wrong right there's another container that gets spun up and your application runs in that particular container very less chances of your application going down that's what i mean this is not possible with a physical host this is very less possible with an vm or at least it's going to take some time for another vm to get spun up so why ec is or what is ecs amazon ecs maintains the availability of the application and allows every user to scale containers when necessary so it not only meets the availability of the application meaning one container running your application or one container hosting your application should be running all the time so to meet that high availability availability is making sure your service is running 24 7. so container makes sure that your services run 24 bar 7 not only that not only that suddenly there is an increase in demand how how do you meet that demand right let's say you have like thousand users suddenly the next week there are like 2000 users all right so how do you meet that demand container makes it very easy for you to meet that demand in case of vm or in case of physical host you literally will have to go buy another physical host or add more ram add more memory add more cpu power to it all right or kind of club two three hosts together clustering you would be doing a lot of other things to meet that high availability and also to meet that demand but in case of uh ecs it automatically scales the number of containers it automatically scales the number of containers needed and it meets your demand for that particular r so what is amazon ecs the full form of ecs is elastic container service right so it's basically a container management service which can quickly launch and exit and manage docker containers on a cluster so it's the function of ecs it helps us to quickly launch and quickly exit and manage docker container so it's kind of a management service for the docker containers you will be running in amazon or running in the aws environment so in addition to that it helps to schedule the placement of container across your cluster so it's like this you have two physical hosts you know joined together as a cluster and ecs helps us to place your containers now where should your container be placed should it be placed in host 1 should be placed in host 2. so that logic is defined in ecs we can define it you can also let ecs take control and define that logic most cases you will be defining it so schedule the placement of containers across your cluster let's say two containers want to interact heavily you really don't want to place them in two different hosts right you would want to place them in one single host so they can interact with each other so that logic is defined by us and these container services you can launch containers using aws management console and also you can launch containers using sdk kits available from amazon you can launch through a java program you can launch container using and.net program you can launch container using an node.js program as in when the situation demands so there are multiple ways you can launch containers through management console and also programmatically and ecs also helps to migrate application to the cloud without changing the code so anytime you think of migration the first thing that comes to your mind is that how will that environment be based on that i'll have to alter my code what's what's the ip what is the storage that's being used what what are the different parameters i'll have to include the environment parameters of the new environment with containers now that worry is already taken away because we can create and pretty exact environment that the one that you had an on-premises the same environment gets created in the cloud so no worries about changing the application parameter no worries about changing the code in the application right you can be like if it ran in my laptop a container that i was running in my laptop it's definitely going to run in the cloud as well because i'm going to use the same container in the laptop and also in the cloud in fact you're going to ship it you're going to move the container from your laptop to amazon ecs and make it run there so it's like the same the very same image the very same container that was running in your laptop will be running in the cloud or production environment so what is docker we know that it ecs helps to quickly launch exit and manage docker containers what is docker let's let's answer that question what is docker now docker is a tool that helps to automate the development of an application as a lightweight container so that the application can work efficiently in different environments this is pretty much what we discussed right before the slide i can build an application in my laptop or in on premises in a container environment docker container environment and anytime i want to migrate right i don't have to kind of rewrite the code and then rerun the code in that new environment i can simply create an image docker image and move that image to that production or the new cloud environment and simply launch it there right so no compiling again no relaunching the application simply pack all your code in a docker container image and ship it to the new environment and launch the container there that's all so docker container is a light weight package of software that contains all the dependencies so because you know when packing you'll be packing all the dependencies you'll be packing the code you'll be packing the framework you'll be packing the libraries that are required to run the application so in the new environment you can be pretty sure you can be guaranteed that it's going to run because it's the very same code it's the very same framework it's the very same libraries that you have shipped right there's nothing new in that new environment it's the very same thing that's going to run in that container so you can be rest assured that they are going to run in that new environment and these docker containers are highly scalable and they are very efficient suddenly you wanted like 20 more docker containers to run the application think of adding 20 more hosts 20 more vms right how much time would it take and compared to that time the amount of time that docker containers would require to kind of scale to that amount like 20 more containers it's very less or it's minimal or negligible so it's an highly scalable and it's a very efficient service you can suddenly scale number of docker containers to meet any additional demand very short boot up time because it takes a it's not going to load the whole operating system and these docker containers you know they use the uh linux kernel and features of the kernel like c groups and name spaces to kind of segregate the processors so they can run independently any environment and it takes very less time to boot up and the datas that are stored in the containers are kind of reusable so you can have an external data volume and i can map it to the container and whatever the space that's occupied by the container and the data that the container puts in that volume they are kind of reusable you can simply remap it to another application you can kind of remap it to the next successive container you can kind of remap it to the next version of the container next version of the application you'll be launching and you don't have to go through building the data again from the scratch whatever data the container was using previously or the previous container was using that data is available for the next container as well so the volumes that the containers users are very reusable volumes and like i said it's isolated application so it kind of isolates by its nature it kind of by the way it's designed by the way it is created it isolates one container from another container meaning anytime you run applications on different containers you can be rest assured that they are very much isolated though they are running on the same host though they are running on the same laptop let's say though they are running on the same physical machine let's say running 10 containers 10 different applications you can be sure that they are well disconnected or well isolated applications now let's talk about the advantages of ecs the advantage of ecs is improved security its security is inbuilt in ecs with ecs we have something called as a container registry you know that's where all your images are stored and those images are accessed only through https not only that those images are actually encrypted and access to those images are allowed and denied through identity and access management policies iam and in other words let's say two container running on the same instance now one container can have access to s3 and the others or the rest of the others are denied access to s3 so that kind of granular security can be achieved through containers when we mix and match the other security products available in amazon like iam encryption accessing it using https these containers are very cost efficient like i've already said these are lightweight processes right we can schedule multiple containers on the same node and this actually allows us to achieve high density on an ec2 instance imagine an ec2 instance that that's very less utilized that's not possible with a container because you can actually dense or crowd an ec2 instance with more container in it so to best use those resources in ec2 straightforward you can just launch one application but with when we use containers you can launch like 10 different applications on the same ec2 server that means 10 different applications can actually feed on those resources available and can benefit the application and ecs not only deploys the container it also maintains the state of the containers and it makes sure that the minimum a set of containers are always running based on the requirement that's another cost efficient way of using it right and anytime an application fails and that has a direct impact on the revenue of the company and is just make sure that you're not losing any revenue because your application has failed and easies and pretty extensible services it's like this in many organization there are majority of unplanned work because of environment variation a lot of firefighting happens when we kind of deploy the code from one or kind of move the core or redeploy the code in a new environment a lot of firefighting happens there right this docker containers are pretty extensible like we discussed already environment is not a concern for containers because it's going to kind of shut itself inside a docker container and anywhere the docker container can run the application will run exactly the way it performed in the past so environment is not a concern for the docker containers in addition to that ecs is easily scalable we have discussed this already and it improves it has improved compatibility we have discussed this already let's talk about the architecture of ecs like you know now the architecture of ecs is the ecs cluster itself that's group of servers running the ecs service and it integrates with docker right so we have a docker registry docker registry is a repository where we store all the docker images or the container images so it's like three components ecs is of three components one is the ecs cluster itself right when i say easy as itself i'm referring to easiest cluster cluster of servers that will run the containers and then the repository where the images will be stored right the repository where the images will be stored and the image itself so container image is the template of instructions which is used to create a container right so it's like what's the os what is the version of node that should be running and any additional software do we need so those question gets answered here so it's the template template of instructions which is used to create the containers and then the registry is the service where the docker images are stored and shared so many people can store there and many people can access or if there's another group that wants to access they can access the image from there or one person can store the image and rest of the team can access and the rest of the team can store image and this one person can pick the image from there and kind of ship it to the customer or ship it to the production environment all that's possible in this container registry and amazon's version of the container registry is ecr and there's a third party docker itself has a container registry that's docker hub ecs itself which is the the group of servers that runs those containers so these two the container image and the container registry they kind of handle docker in an image format just an image format and in ecs is where the container gets live and then it becomes and compute resource and starts to handle requests and starts to serve the page and starts to do the batch job you know whatever your plan is with that container so the cluster of servers ecs integrates well with the familiar services like vpc vpc is known for securing vpc is known for isolating the whole environment from rest of the customers or isolating the whole environment or the whole infrastructure from the rest of the clients in your account or from the rest of the applications in your account on and on so vpc is a service that provides or gives you the network isolation ecs integrates well with vpc and this vpc enables us to launch aws resources such as amazon ec2 instance in a virtual private network that we specify this is basically what we just discussed now let's take a closer look at the ecs how does ecs work let's find answer for this question how does ecs work ecs has got a couple of components within itself so these ecs servers can run across availability zone as you can see there are two availability zones here they can actually run across availability zones and ecs has got two modes far gate mode and the ec2 mode right here we're seeing forget mode and then here we see nothing that means it's an ec2 mode and then it has got different network interfaces attached to it because they need to be running in an isolated fashion right so anytime you want network isolation you need separate ip and if you want separate ip you need separate network interface card and that's what you have elastic network interface card separate elastic network interface card for all those tasks and services and this runs within an vpc let's talk about the far gate service tasks are launched using the far gate service so we will discuss about uh task what is forget now fargate is a compute engine in ecs that allows users to launch containers without having to monitor the cluster ecs is a service that manages the containers for you right otherwise managing containers will be and full-time job so easy is manages it for you and if you and you get to manage ecs that's the basic service but if you want amazon to manage ecs and the containers for you we can go for fargate so farget is a compute engine in ecs that allows users to launch containers without having to monitor the ecs cluster and the tasks the tasks that we discussed the tasks has two components you see task right here so they have two components we have ecs container instance and then the container agent so like you might have guessed right now ecs container instance is actually an ec2 instance right capable of running containers not all ec2 instances can run containers so these are like specific ec2 instances that can run containers they are ecs container instances and then we have container agent which is the agent that actually binds those clusters together and it does a lot of other housekeeping work right kind of connects clusters makes sure that the version needed is present so it's all part of that agent or it's all job of that agent container instances container instances is part of amazon ec2 instance which run amazon ecs container agent pretty straightforward definition and then a container agent is responsible for communication between ecs and the instance and it also provides the status of the running containers kind of monitors the container monitors the state of the container make sure that the content is up and running and if there's anything wrong it kind of reports it to the appropriate service to fix the container on and on it's a container agent when we don't manage container agent it runs by itself and you really don't have to do anything to make the container agent better it's already better you really won't be configuring anything in the agent and then elastic network interface card is a virtual interface network that can be connected uh to an instance in vpc so in other words elastic network interface is how the container interacts with another container and that's how the container interacts with the ec2 host and that's how the container interacts with the internet external world and a cluster a cluster is a set of ecs container instances it's not something that's very difficult to understand it's simply a group of ec2 instances that runs that ecs agent and this cluster in cluster handles the process of scheduling monitoring and scaling the request we know that ecs can scale the containers can scale how does it scale that's all monitored and managed by this ecs cluster let's talk about the companies that are using amazon ecs there are variety of companies that use ecs clusters to name a few octa users easiest cluster and octa is a product that use identity information to grant people access to applications on multiple devices at any given point of time they make sure that they have a very strong security protection so octa uses amazon ecs to run their octa application and serve their customers and abima abhima is an uh tv channel and they chose to use microservices and docker containers they already had microservices and docker containers and when they thought about a service that they can use in aws ecs was the only service that they can immediately adapt to and because in abima tv the engineers have already been using docker and docker containers it was kind of easy for them to adapt themselves to ecs and start using it along with the benefits that ecs provides previously they had to do a lot of work but now ecs does it for them all right similarly remind and ubisoft gopro are some of the famous companies that use amazon ecs and get benefited from its scalability get benefited from its cost gets benefited from its amazon managed services get benefited from the portability that ecs and the migration option that ecs provides let's talk about how to deploy a docker container on amazon acs the way to deploy docker container on ecs is first we need to have an aws account and then set up and run our first ecs cluster so in our lab we're going to use the launch wizard to run an ecs cluster and run containers in them and then task definition task definition tells the size of the container the number of the container and when we talk about size it tells how much your cpu do you need how much of memory do you need and talking about numbers you know it requires how many numbers of container you're going to launch you know is it 5 is it 10 or is it just one running all the time now those kind of information goes in the task definition file and then we can do some advanced configuration on ecs like load balancers and you know what port number you want to allow and you don't want to allow you know who gets access who shouldn't get access and what's the ip that you want to allow and deny requests from on and on and this is where we would also mention the name of the container so to differentiate one container from the other and the name of the servers you know is it an a backup job is it a web application is it an uh data container is it going to take care of your data data backend and the desired number of tasks that you want to be running all the time those details go in when we try to configure the ecs service right and then you configure cluster you put in all the security in the configure your cluster step or configure cluster stage and finally we will have an instance and bunch of containers running in that instance all right let's do a demo so here i have logged in to my amazon portal and let me switch to the appropriate region i'm going to pick not virginia north virginia look for ecs and it tells ecs as a service that helps to run and manage docker containers well and good click on it i'm in north virginia just want to make sure that i'm in the right region and go to clusters and here we can create cluster this is our far gate and this is our ec2 type launching for linux and windows environment but i'm going to launch through this walkthrough portal right this gives a lot of information here so the different steps involved here is creating a container definition which is what we're going to do right now and then a task definition and then service and finally the cluster it's a four step process so in container definition we define the image the base image we are going to use now here i'm going to launch an uh httpd or a simple http webpage right so a simple httpd 2.4 image is fair enough for me and it's not a heavy application so 0.5 gigabit of memory is enough and again it's not a heavy application so 0.25 virtual cpu is enough in our case right you can edit it based on the requirement you can always edit it and because i'm using hdpd the port mapping is already port 80 that's how the container is going to receive the request and there's no health check as if now when we want to design critical and complicated environments we can include health check right and this is the cpu that we have chose we can edit it and i'm going to use some bash commands to create an html page right this page says that you know amazon ecs sample app right and then it says amazon ecs sample app your application is running on a container in amazon ecs so that's the page the html page that i'm going to create or index.html so i'm going to create and put it in an appropriate location so those pages can be served from the container right if you replace this with any of your own content then it's going to be your own content ecs comes with some basic logs and these are the places where they get stored that's not the focus as of now all right so i was just saying that you can edit it and customize it to your needs we're not gonna do any customization now we're just getting familiar with ecs now and the task definition name of the task definition is first one task definition and then we are running it in a vpc and then this is and for gate mode meaning the servers are completely handled by amazon and the task memory is 0.5 gigabit and the tar cpu is 0.25 virtual cpu name of the service is it a batch job is it an you know front end is it a back end or is it a simple copy job what's the service name of the service goes here again this you can edit it and here's a security group as of now i'm allowing 480 to the whole world if i want to restrict to a certain ip i can do that the default option for load balancing is a no load balancer but i can also choose to have a load balancer and use port 80 to map that 480 to the container put 80 right i can do that the default is no load balancer all right let's do one thing let's use load balancer let's use load balancer and port 80 as it receives information on port 80 http uh what's going to be the cluster name we're in the last step what is the cluster name cluster name can be simply learn ecs demo next we're done and we can create so it's launching a cluster as you can see and it's picking the task definition file that we've created and it's using that to launch and service and then these are the log groups that we discussed and it's creating your vpc remember ecs clubs well with the vpc it's creating a vpc and it's creating two subnets here for high availability it's creating that security group port 80 allowed to the whole world and then it's putting it behind and load balancer it generally would take like five to 10 minutes so we just need to be patient and let it complete its creation and once this is complete we can simply access these servers using the load balancer url and when this is running let me actually take you to the other products or the other services that are integrated with the ecs it's getting created our service is getting created as of now ecr repository this is where all our images are stored now as of now i'm not pulling my image from ecr i'm pulling it directly from the internet docker docker hub but all custom images all custom images they are stored in this repository so you can create a repository call it app one create a repository so here's my repository so any image that i create locally or any docker image that i create locally i can actually push them push those images using these commands right here and they get stored here and i can make my ecs connect with ecr and pull images from here so they would be my custom images and as of now because i'm using a default image it's directly pulling it from the internet let's go to ec2 and look for a load balancer because we wanted to access the application from behind a load balancer right so here is a load balancer created for us and anytime i put the url so cluster is now created you see there's one service running all right let's click on that cluster here is the name of our application and here is the tasks the different containers that we are running and if you click on it we have an ip right ip of that container and it says it's running it was created at such and such time and started at such and such time and this is the task definition file that it this container uses meaning the template the details to all the version details they all come from here and it belongs to the cluster called simply learn ecs demo right and you can also get some logs container logs from here so let's go back and there are no ecs instances here because remember this is forget you're not managing any ecs instance all right so that's why you're not seeing any ecs instance here so let's go back to tasks and go back to the same page where we found the ip pick that ip put it in the browser and you have this sample html page running from an container so let me go back to load balancer ec2 and then under ec2 i'll be able to find a load balancer find that load balancer pick that dns name put it in the browser and now it's accessible to the load balancer url right now this url can be mapped to other services like dns this url can be embedded in any of your application if you want to make that application connect with this container now using ip is not all that advisable because these containers can die and then a new container gets created and when a new container gets created it gets a new ip right so a hard coding ip is not hardcoding dynamic ips are not advisable so you would be using load balancer and putting that url in that application that you want to make it interact with this container instance it was a wonderful experience in walking you through this ecs topic and in here we learned about what aws is and why we're using ecs and what is easiest in general what is docker in specific and we also learn about the advantages of ecs the architecture the different components of ecs and how ecs works when they're all connected together and we also looked at the companies that use ecs and their use cases and finally a lab how we can launch ecs fargate through the portal i'm very glad to walk you through this lesson about route 53. so in this section we are going to talk about basics of aws and then we're going to immediately dive into why why we need amazon route 53 and then we're going to expand and talk about the details of amazon route 53 the benefits it provides over its competitors and the different types of routing policy it has and some of amazon ralph 53's key features and we're going to talk about how to access route 53 i mean the different ways the different methods you can access route 53 and finally we're going to end with an a wonderful demo in drought 53. so let's talk about what is aws amazon web services or aws in short is a cloud provider that offers a variety of services such as a variety of id services or infrastructure services such as a compute power database content delivery and other resources that helps us to scale and grow our business and aws is hot aws is picking up aws is being adapted by a lot of customers that's because aws is easy to use even for a beginner and talking about safety the the aws infrastructure is designed to keep the data safe irrespective of the size of the data with small data be it very minimal data be it all the data that you have in terabytes and in petabytes amazon can keep it safe in their environment and the wonderful thing and the most important reason why a lot of customers move into the cloud is that the pay as you go pricing there is no long term commitment and it's very cost effective what this means is that you're not paying for resource that you're not using in on-premises you do pay for resources you're not using a lot meaning you go and buy a server you do the estimate for the next five years and only after like three or four years you'll be hitting the peak capacity but still you would be buying that capacity before four years right and then you will gradually be you know utilizing it from you know 40 days 60 days 70 80 and then 100. so what you have done is that even though you're not using the full capacity you still have bought it and are and are paying for it from day one but in the cloud it's not like that you only pay for the resources that you use anytime you want more you scale up the resource and you you pay for the scaled up resource and anytime you want less you scale down the resource and you pay less for that scaled down resource let's talk about why amazon route 53 let's take this scenario where rachel is trying to open her web browser and the url that she hit isn't working a lot of reasons behind why the url isn't working it could be the server utilization that went high it could be it could be the memory usage that went high a lot of reasons and she starts to think is there an efficient way to scale resources according to the user requirements or is there an efficient way to kind of mask all those failures and kind of divert the traffic to the appropriate active you know active resource or active service that's running our application you always want to hide the failures right in it kind of mask the failure and direct the customer to another healthy service that's running right none of your customers would want to see a server not available or you know none of the customers your customers would want to see your service not working not impressive to them and this is tom tom is an i.t guy and he comes with an idea and he's answering rachel yes we can scale resources efficiently using amazon rod53 in a sense he's saying that yes we can mask the failure and we can keep the services up and running meaning we can provide more high availability to our customers with the use of route 53 and then he goes on and explains amazon rough 53 is a dns service that gives developers an efficient way to connect users to internet applications without any downtime now downtime is the key amazon route 53 helps us to avoid any downtime that customers will experience you still will have downtime in your server and your application but your customers will not be made aware of it and then rachel is kind of interested and she's like yeah that sounds interesting i want to learn more about it and tom goes on and explains the important concepts of amazon route 53 that's everything that i'm going to explain it to you as well all right so what is amazon rat53 amazon drop 53 is a highly scalable dns or domain name system web service this service this amazon raw 53 it functions three main things or it has three main functions so the first thing is if a website needs a name route 53 registers the name for the website domain let's say you want to buy google.com you want to buy the domain name let's say you want to buy that domain name you buy that through rot 53 secondly a route 53 is the service that actually connects your server which is running your application or which is holding which is serving your web page so that's the service that actually route 53 is the service that connects the user to your server when they hit google.com in the browser or whatever domain name that you have purchased so you bought a domain name and the user types in yourdomainname.com and then roth53 is a service that helps the user to connect their browser to the application that's running in an ec2 instance or any other server that you are using to serve that content and not only that 53 checks health of the resource by sending automated requests over the internet to a resource so that's how it identifies if there is any resource that has failed can i say resource i'm referring to any infrastructure failure any application level failure so it kind of keeps checking so it understands it first before the customer notices it and then it does the magic kind of shifts the connection from a one server to the other server we call it routing we will talk about that as we progress so the benefits of using route53 it's highly scalable meaning suddenly let's say the number of requests the number of people trying to access your website through that domain name that you have bought let's say it has increased ralph 53 is highly scalable right it can handle even millions and millions of requests because it's highly scalable and it's managed by amazon the same thing it's reliable it's a highly scalable it can handle large queries without the users without you interacting without the user who bought it interact with it you don't have to scale up you know when you're expecting more requests it automatically scales and it is very reliable in a sense that it's very consistent it has the ability to route the users to the appropriate application through the logic that it has it's very easy to use when we do the lab you're going to see that it's very easy easy to use you buy the domain name and then you simply map it to the application you simply map it to the server by putting in the ip or if you you can simply map it to another load balancer by putting in the load balancer url you can simply map it to another s3 bucket by simply putting the s3 bucket name or the s3 bucket url it's pretty straightforward easy to set up and it's very cost effective in a way that we only pay for the service that we have used so no wastage of money here so the billing is set up in such a way that you are paying only for the amount of requests that you have received right the amount of traffic the amount of requests that you have received and couple of other things the the number of uh hosted zones that you have created right and couple of other things it it's very cost effective in such a way that you only pay for the service that you are using and it's secure in a way that access to route 53 is integrated with identity and access management iam so you only have authorized users gain access to route 53 the trainee who just joined yesterday won't get access and the contractor or the consultant the third party consultant you have given access or who is using your environment you can block access to that particular person because he's not the admin or he's not a privileged user in your account so only privileged users and admin gain access to route 53 through iam now let's talk about the routing policies so when you create a record in in route 53 recorder is nothing but an entry so when you do that you choose a routing policy right routing policy is nothing but it determines how route 53 responds to your queries how the dns queries are being responded right that's that's a record or that's a routing policy so the first one is a simple routing policy so we use simple routing policy for a single resource in other words simple routing allows to configure dns with no special route 53 routing it's kind of one to one you use an single resource that performs a given function to your domain for example if you want to simply map an url to a web server that's pretty straightforward simple routing so it routes traffic to a single resource example web server to a website and with simple routing multiple records with the same name cannot be created but multiple values can be created in the same record the second type of routing policy is failover routing so we would be using failover routing when we want to configure active passive failover if something failed right you want to fail over to the next resource which was previously the backup resource now the active resource or which was previously the backup server now it's an active server so you would be failing over to that particular resource or that particular ip if you want to do that we use failover routing so failover routing routes traffic to a resource when the resource is healthy or to a different resource when the previous resource is unhealthy in other words anytime a resource goes unhealthy i mean it does all that's needed to shift the traffic from the primary resource to the secondary resource in other words from the unhealthy resource to the healthy resource and this records can draw traffic to anything from an amazon s3 bucket or you can also configure a complex tree of records now when we configure the records it will be more clear to you so as of now just understand that route 53 can route or this routing policy the failover routing policy can route traffic to amazon s3 bucket or to a website that has complex tree of uh records geolocation routing policy now geolocation routing just like the name says it takes that routing decision based on the geographic location of the user in other words you know when you want to roll traffic based on the location of the user that's your primary criteria for you know sending that request to the appropriate server we will be using geo location routing so it localizes the content and presents a part or the entire website in the language of the user for example a user from us you would want to direct them to an english website and a user from german if you want to send them to the german website and a user from france you know you want to send those requests or you want to show content specific to a customer who lives in france a french website so this is if that's your condition this is the routing policy we would be using and the geographic locations are specified by either continent or by country or by state in the united states so only in the united states you can actually split it to state level and for the rest of the countries you can do it on a country level on a high level you can also do it on a continent level the next type of routing policy would be job proximity routing geoproximity routing policy when we want to route traffic based on the location of our resource and optimally shift traffic from resources in one location to resource in another location we would be using geoproximity routing so geoproximity routing routes traffic to the resources based on the geographic location of the user and the resources they want to access and it also has an option to route more traffic or less to a given resource by specifying a value known as a bias kind of weight but we also have weighted routing that's different so we've chosen different name bias you can send more traffic to a particular resource by having a bias on that particular routing condition and a bias expands or shrinks the size of the geographic region from which traffic is routed to a resource and then we have latency based routing just like the name says we use latency based routing if we have resources in multiple aws regions and if you want to route traffic to the region that provides the best latency at any given point of time so let's say if one single website needs to be installed and hosted on multiple aws regions then latency routing policy is what is being used it improves the performance of the users by serving the request from the aws region that provides the lowest latency so at any given point if performance is your criteria and at any given point of time irrespective of what happens in amazon infrastructure irrespective of what happens in the internet if you want to route your users to the best performing website best performing region then we would be using latency based routing and for using latency-based routing we should create latency records for the resources in multiple aws regions and then the other type of a routing policy is a multi-value routing policy where we can make route 53 to respond to dns queries with up to eight healthy records selected at random so you're not kind of loading one particular server we can define eight records and on a random basis route 53 will respond to queries from these eight records so it's not one server that gets all the requests but eight servers gets the request in a random fashion so it's multi-value routing policy and what we get by this is that we are distributing the traffic to many servers instead of just one server so multi-value routing configures route 53 to return multiple values in response to a single or multiple dns queries it also checks the health of order sources and returns the multiple values only for the healthy resources let's say out of the eight servers we have define one server is not doing healthy it will not respond to the query with the details of the unhealthy server right so now it's gonna treat it as only seven servers in the list because one server is unhealthy and it has the ability to return multiple health checkable ip addresses to improve availability and load balancing the other type of routing policy is weighted routing policy and in here we use to route traffic or this is used to route traffic to multiple resources in a proportion that we specify so this is an weighted routing and weighted routing routes multiple resources to a single domain name or a sub domain and control the traffic that's routed to each resources so this is very useful when you're doing load balancing and testing new versions of the software so when you have a new version of the software you really don't want to send 100 of the traffic to it so you want to get customers feedback about the new software that you've launched new version or new application that you've launched so you would kind of send only 20 of the traffic to that application get customer feedback and if all is good then we would move the rest of the traffic to that new application so any software launches application launches will be using weighted routing now let's talk about the key benefits or key features of route 53 some of the key features of route 53 are traffic flow it routes end users to the endpoint that should provide the best user experience that's what we discussed in the routing policies right it uses a routing policy a latency based routing policy and jio based routing policy and then failover routing policy so it kind of improves the user experience and the key feature the other key feature of route 53 is we can buy domain names using roth 53 using route 53 console we can buy it from here and use it in a rough 53. previously it was not the case but now we can buy it directly from amazon through route 53 and we can assign it to any resources that we want so anybody browsing that url the connection will be directed to the server in aws that runs our website a health checks it monitors health and performance of the application so it comes with an health check attached to it health check are useful to make sure that the unhealthy resources are retired right the unhealthy resources are taken away or your customers are not kind of hitting the unhealthy resources and they see an service down page or something like that uh we can have weighted round robin load balancing that's helpful in spreading traffic between several services or servers we are round robin algorithm so no one server is fully hit or no one server kind of fully absorbs all the traffic you know you can shift you can split and shift the traffic to different servers based on the weight that you would be configuring and also weighted routing also helps with a soft launch soft launch of your new application or the new version of your website there are different ways we can access amazon route 53 so you can access amazon rot 53 through aws console you can also access amazon route 53 using aws sdks and we can access it using we can configure it using the apis and we can also do it through the command line interface that's linux type linux flavor aws command line interface we can also do that using windows command line windows powershell flavored command line interface as well now let's look at some of the companies that are using ros 53 so some of the famous companies that use ralph 53 are medium medium is an online publishing platform and it's more like a social journalism it's kind of having hybrid collection of professionals people and publications or exclusive blogs or publishers on medium it's kind of an blog website and that uses roth53 for the dns service reddit is an uh social news aggregation or web content rating and discussion website that uses route 53 so these are some websites that that are accessed throughout the world and they are using roth 53 and it's highly scalable suddenly there is a new news right their website will be accessed a lot and they need to keep their service up and running all the time more availability otherwise customers will end up in a broken page and the number of customers who will be using the website will come down so it's very critical these sites these companies are very critical you know they're being highly available their page their site being highly available and the internet is very critical and crucial for them and they rely and use route 53 to meet that particular demand and airbnb is another company uh instacart cozara is another company stripe is another company that uses route53 to as their dns uh provider for their dns service they use route 53 so their customers get best performance they use raw 53 so their website is highly available they use rot 53 to kind of shift to the traffic between the resources so their resources are properly used with all the weighted routing the resources are properly used now let's quickly look at a demo i'm in my aws console and i'm in route 53 so let me click on rot53 so in this lab we're actually going to simulate buying a domain name and then we're going to create an s3 static website and we're going to map that website to this dns name right so the procedure is the same for mapping load balancer the procedure is the same for mapping cloudfront the procedure is the same for mapping ec2 instances as well we're picking s3 for simplicity right but our focus is actually on route53 so let's go in here and we'll see if we can we'll buy a domain name here so let's first check the availability of a domain name called simply learn hyphen demo hyphen route 53 let's check its availability it is available for 12 dollars so let me add it to cart and then come back here and then once you continue to ask for personal information once you give the personal information you finally check out and then it gets added to your shopping list once you pay for it amazon takes like 24 to 40 hours to make that a dns name available so the next stage would be contact details and then the third stage would be verify and purchase so once we have bought the domain name it will become available in our dns portal and i do have a domain name which i bought some time back and it's now available for me to use so i can go to hosted zone and simply start creating i can go to hosted zone and then here it's going to list all the domain names for me right click on the domain name and then click on the record set and here i can actually map elastic load balancer s3 website vpc endpoint api gateway and cloudfront elastic bean stock domain names right all that gets mapped through this portal quite simple like four or five step button clicks and then it'll be done so i have an domain name bot and then i'm gonna go to s3 and i'll show you what i've done in s3 so i've created a bucket name called as a dns name let me clear the content in them so i've created a bucket and then permissions i've turned off public access blocking and then i've created and a bucket policy so this bucket is now publicly accessible and then i went on to properties and created the static website hosting right and i've pointed that this is the file that's my index file that i'm going to put or name of the file that's going to be my index file that i'm going to put in this s3 bucket so put the index file.html saved it and we're going to create a file now we're going to create an index file so this is a sample code it says amazon.53 getting started routing internet traffic to s3 bucket for your website and then a couple of other information so save it as an index.html file in my desktop so let me upload that from my desktop into this bucket so that's index.html and it's in capital i so let me go to properties and go to static website hosting and make sure that i spell it properly right it's case sensitive and then save it so now this means that my website should be running through this url and it does it's running to the static website url we're halfway through so now let me go back to route53 go back to route53 go back to hosted zones go into the domain name and then create a record set and it's going to be an alias record and i i see my s3 static website endpoint there right so click on it and create it has now created an record that's pointing my domain name to the s3 endpoint that i have created and my static website is running from it so let me test it right so let me go to the browser put the domain name in there and sure enough the domain name when my browser queried for the domain name a route53 returned a response saying this domain name is actually mapped to the s3 bucket static website hosting enable s3 bucket and this is the url for that static website hosting and then my browser was able to connect to that s3 bucket and download the details and show it in my browser right so it's that simple and pretty straight forward today's session is on aws elastic beanstalk so what's in it for you today we'll be discussing about what is aws why we require aws elastic bean stock what is aws elastic bean stock the advantages disadvantages the components of mean stock along with that the architecture and the companies that are primarily using the aws be installed so let's get started and first understand what is aws aws stands for amazon web services it's a cloud provider and that offers a variety of services such as compute power database storage content delivery and many other resources so we know that aws is the largest cloud provider in the market and so many services are available in the aws where you can apply the business logics and create the solutions using the cloud platforms now why aws elastic means stock now what happened earlier and that whenever the developer used to create the software or the modules related to the softwares has to be joined together to create a big application now one developer creates a module that has to be shared with another developer and if the developers are geographically separated then it has to be shared over a medium probably an internet so that is going to take some time uh it would be a difficult process and in return it uh makes the application or a software development a lender process the building of the software development lender process so there were challenges which the developers were facing earlier and to overcome that uh we have the bean stock as a service available in the aws so why aws elastic bean stock is required uh aws elastic being stock has made the life of the developers quite easy in terms of that they can share the applications across different devices at a shorter time duration now let's understand what is aws elastic bean stock aws elastic bean stock is a service which is used to deploy and scale web applications by developers not only web application any application that is being developed by the developers this is a simple representation of the aws elastic bean stock now along with that the aws elastic bean supports the programming language the runtime environments that are java.net php node.js python rubygo and docker and in case if you're looking for any other programming language or a runtime environment then you can make a request with aws to arrange that for you now what are the advantages associated with the elastic bean stock first advantage is that it's a highly scalable service now when we talk about the scalability it means that whenever we require the resources in demand we can scale up the resources or we can scale down the resources so that is kind of a flexibility we get in terms of changing the type of resources whenever we need it and in that case the elastic bean stock is a highly scalable service now that is something which is very difficult to achieve in case of an on-prem environments because you have to plan for the infrastructure and in case if you're short of the resources within that infrastructure then you have to procure it again the second advantage associated with the bean stock is that it's a fast and simple to begin now when we say it's fast and simple that means that you just have to focus on the development of an application building an application and then you can just deploy the application directly using the bean stock but the bean stock is going to do that every networking aspect is being taken care by the bean stock it deploys your application in the back end on the servers and then you can directly access your application using the url or through the ip address the third advantage is that it offers the quick deployment that is what we discussed in the fast and simple to begin as well so why it offers a quick deployment you don't have to bother about the networking concepts you just have to focus on the application development and then you can just upload your application deploy that and then you're good to go the other advantage is that it supports multi-tenant architecture when we talk about tenants or multi-tenants that means we can have a virtual environments for separate organizations or the divisions within the organizations that will be virtually isolated so likewise you can have a virtually isolated environments created on the bean stock and they can be separated used as a separate entities or a separate divisions within the organization and we know that it's a flexible service since it's a scalable then it is a flexible also now coming to the simplifies operations as an advantage now once the application is deployed using the bean stock then it becomes very easy to maintain and support that application using the bean stock services itself and the last advantage that we can have from the bean stock is that it's a cost efficient service the cost efficient as we know that many of the aws services are cost effective the cost optimization can be better managed using the aws main stock as compared to if you are developing or if you are deploying any kind of an application or a solution on the on-prem servers now there are some components that are associated with the aws bean stock and it has to be created in the form of a sequence manner so aws elastic bean stock consists of few important components which are required while developing an application now what are these components these are four components one is application the second is application version the third is environment and the fourth one is the environment tier and we have to progress while deploying our applications or the softwares using the same sequence now let's understand what are the different components of the beanstalk r the application it refers to a unique label which is used as a deployable code for a web application so generally you deploy your web application or you create your application and that is something which is basically uh used as a unique label then the second component is application versions so it resembles a folder which stores a collection of components such as environments versions and environment configurations so all these components are being stored using the application version the third most important component is the environment in the environment only the current versions of the applications runs now remember that elastic means stock supports multiple versions as well and using the environment you can only run the current version of the application for if you wanted to have another version of an application to be running then you have to create another environment for that then comes the environment here and in the environment here it is basically it designates the type of application that the environment runs on now generally there are two types of environment here one is the web and the other one is the worker node and that's something which we'll be discussing later as well now let's understand how does elastic beam stock in aws works so first we have to create an application and this is a task that would be done by the developers and for that you can actually select any runtime environment or a programming language like java docker ruby gopal or python as well and once you select that environment you can develop your application using that runtime environments now after that once the application is created then you have to upload the version of an application on the aws and after that once the version is uploaded and then you have to launch your environment so just have to click on the buttons that's it nothing more you have to do once the environment is launched then you can actually view that environment using a web url or using the ip address now what happens in that case is when you launch an environment in the back end the elastic bean stock runs automatically runs an ec2 instance and using a metadata the mean stock deploys your application within that ec2 instance that is something which you can look into the ec2 dashboard as well so you don't have to take care of the security groups you don't have to take care of the ip addressing and even you don't have to login into the instance and deploy your application it would be done automatically by the bean stock it's just that you just have to monitor the environment and the statistics will be available there itself in the bin stock dashboard otherwise you can view those statistics in the cloudwatch logs as well now in case if you wanted to update any kind of a version then you just upload a new version and then just deploy that and then monitor your environment so these are the essentials to create a local applications for any platform whether it's a node.js python etc these are the things that you have to actually take care and this is the sequence you have to follow while creating an environment so you can say that it's a four steps creation offer or deployment of your application that's it now after users upload their versions the configuration is automatically deployed with a load balancer yes and with the load balancer that means you can access the applications using the load balancer dns also and apart from load balancer if you wanted to put any other feature that includes the auto scaling for example if you wanted to create your ec2 instances where the application will be deployed within the virtual private cloud or in a particular subnet within the vpc all those features that are available and you can select them using the mean stock itself you don't have to move out to the vpc you don't have to actually go to the ec2 dashboard and select all those separately everything would be available within the beanstalk dashboard so that's what it says in the presentation that after creating an application the deploy service can be specifically accessed using the url so once the environment is created there will be a url defined now you can put a url name also that is something which you wanted to put for your application you can define that you can check for the availability of that url and then you have to use that url to access your application or the browser now once it is done then in the monitor environment it says the environment is monitored provided capacity provisioning load balancing auto scaling and multi features all those features are available there itself in the mean stock now let's understand the architecture of aws elastic bean stock now there are two types of environments that you have to select you can select one is the web server environment and the other one is the worker environment so based on the client requirement bean stock gives you two different types of environment that you have to select generally the web server environment is the front end facing that means the client should be accessing this environment directly using a url so mostly web applications are deployed using that environment the worker environment is the back end applications or on the micro apps which are basically required to support the running of the web applications now it depends on the client requirement what kind of an environment you wanted to select now in the web server environment it only handles the http request from the client so that's why we use the web server environment mostly for the web applications or any application which works on the http https requests so it's not only the http you can use the https as well the worker environment it process background task and minimizes the consumption of resources so again it is just like a kind of a micro service or an application services that are running in the back end to support the web server environment now coming to the understanding of the aws main stock so this is how the architecture of the aws bean stock is designed and you can refer to that image also now in the web server environment let's say if we select a web server environment and it says that if the application receives client request the amazon route 53 sends this request to the elastic load balancer now obviously we discussed here that the web server environment is primarily an environment which receives the http request it's a kind of a client-facing environment now if the application receives a client request amazon from the amazon route 53 this route 53 is a service which is primarily used for dns mapping it's a global service and it may route you can route the traffic from the route 53 matching your domains towards the load balancer and from the load balancer you can point that traffic to the web server environment obviously the web server environment is nothing it's just the ec2 instances that would be running in the back end now here in the diagram you can see that there are two web server environments and they are created in the auto scaling group that means there is some kind of scaling options that are defined as well and these instances are created in an availability zone or they can be created in a different availability zone also for the redundancy as well and these web application servers are further connected to your databases which primarily will be in a different security groups probably it can be an rds database also so all these functionalities all these features are basically available on the elastic mean stock dashboard itself now what happens in that case is if the application receives client requests amazon route 53 send these requests to the load balancer later the load balancer shares those requests among the ec2 instances how does that happen it happens using a predefined algorithm the equal distribution of a load is distributed to both the ec2 instances or n number of ac2 instances running in the availability zone now in the availability zones every ec2 instance would have its own security group they can have a common security group also they can have their own security group as well now after the security group the load balancer is then connected to the amazon ec2 instance which are part of the auto scaling group so that's something which we have discussed already now this auto scaling group is would be defined from the beanstalk itself and there will be some scaling options that will be created it could be a possibility that it might be the minimum number of instances that would be running as of now and based on the threshold defined it may increase the number of ec2 instance and the load balancer will keep on distributing the load to as many instances that will be created inside the availability resource obviously there will be an internal health check that the load balancer will be first doing before distributing the real time traffic to this instances created by the mean stock now what does auto scaling group does it automatically starts the additional ec2 instance to accommodate increasing load on your application that's something which we know that and also it monitors and scales instances based on the workload as well so depends on what kind of a scaling threshold you have defined in the auto scaling groups and when the load of an application decreases the ec2 instance will also be decreased so whenever we talk about the auto scaling generally it comes in our mind is that we scale up the resources that means we it increases the ec2 instances in the auto scaling you might have the scale down option also scaled down policy also created in which if the load minimizes it can terminate the additional ec2 instances as well so that is something which will be automatically managed all these features can be achievable using the elastic bean stock and with this feature accommodated it gives you the better cost optimization in terms of managing your resources now it says that elastic bean stock has a default security group and the security group acts as a firefall for the instances now here in this diagram it says about the security group auto scaling also you might create it in a default vpc also you might create it in your custom vpc also where you can have the additional level of security is also created you can have the nacl's knuckles also defined here before the security groups so that would give you the additional filtering option or the firewall option now it says that with these groups with these security groups it allows establishing security groups to the database server as well so every database would also have its own security group and the connection can be created between the web servers environment that is created by the beanstalk to the database security groups as well now let's discuss about the worker environment now understanding the worker environment what happens is that the client the web server environment is the client facing the client sends a request for an access to the web server and in this diagram the web server further sends it to the sqs which is a simple queue service and the queue service sends it to the worker environment and then whatever the worker environment is created for doing some kind of a processing or some kind of an application that is running in the back end that environment initiates and then send back uh the results to this sqs and vice versa so let's understand the architecture of a aws elastic bean stock with the worker environment so when a worker environment here is launched aws elastic main stock installed the server on every ec2 instance so that is in the case of a web server environment also and later the server passes the request to the simple queue service now this service is an asynchronous service instead of a simple queue service you can have other services also it is not necessary that you need to have the sqs also this is an example that we are discussing about and the sqs shares those message via a post request to the http path over the broker environment and there are many case studies also with respect to this kind of an environment that is being created that has been done on many customers and you can search for these kind of a case studies available on the internet now the worker environment executes the task given by the sqs with the http response after the operation is completed now here what happens is a quick recap the client request for an access of an application to a web server using an http request the web server passes that request to the queue service the q service shares the message with the worker probably a worker might be the manual worker and generally it's an automated worker so it would be shared by the worker environment only and the worker sends back the response with the http response back to the queue that response can be viewed directly from the queue service by the client using the web server so this is one of the example likewise as i said that there can be many other examples also where you can have the worker environments defined now what are the companies that are using the elastic bean stock these are few of the companies that are primarily using on a zillow jelly button games then you have league of women waters ebery these are some of the few listed companies and obviously you search on the aws site and you will find many more organizations that are using the elastic bean stock primarily for deploying their applications now the next thing is to go with the practicals that how actually we use the elastic bean stock so let's look into the demo using the aws elastic bean stock now first you have to login into the aws console and i'm sure that you might be having the accounts created or you can use the im credentials as well and then you have to select the region also now i am in the north virginia region likewise you can select any of the regions that are listed here now click on the services and you have to search for the elastic bean stock you can find the elastic bean stock under the compute section so here itself you'll find the elastic bean stock as a service now open this service and there it will give you an option to create an environment you have to specifically select an environment probably a worker environment or a web service environment so let's wait for the service to open so we have the dashboard now available with us this is how the elastic bean stock looks and this is the symbol representation of a bean stock now what you have to do is we have to click on get started and that will load and you have to create a web app so instead of creating a web app what we'll do we'll create a new application so just click on create a new application put an application name let's say we put something like x y z you can put any description to your application let's say it's a demo app and click on create now it says you have to select an environment now the environment the application name xyz is created you just have to select the nav environment so click on create one now and it is going to ask you that what kind of an environment here you wanted to select so as we discussed that there are two types of environments one is the web server and the other one is the current one let's look into it what is defined by the aws aw says that it has two types of environment tiers to support different types of web applications web servers are standard applications that listen for and then process http request typically over port number 80. workers are specialized application that have a background processing task that listens for message on an amazon sqs queue workers application post those messages to your application by using the http response so that's what we saw in the case of the beanstalk slides also now the usability of a worker environment can be anything now we'll do a demo for creating a web server environment so just click on select and you we have the environment name created now we can define our own domain it ends with the region dot elastic beanstalk.com let's say i look for a domain which is xyz only that's the environment name now i'll check for the availability whether that domain name is available with us or not and it says we don't have that domain name so probably i'll try to make it with some other name and let's look for the availability xyz abc and it says yes it is available now once i deploy my application i would be able to access the application using this complete dns so you can put a description it's a demo app that we are creating and then you have to define a platform as well now these are the platforms that are supported by the aws let's say i wanted to run a node.js environment so i'll just click on the node.js platform the application codes is something which is basically developed by the developers and you can upload the app application right now or you can do that later as well once the environment is ready now either you can select to create an environment if you wanted to go with all the default settings otherwise if you wanted to customize it more you can click on configure more options so let's click on configure more options and here you would be able to define various different features like the type of an instance for example what kind of an ec2 instance or a server that should be running so that the bin stock can deploy our applications over it if you wanted to modify just click on a modify button and here you can modify your instances with respect to the storage as well now apart from that if you wanted to do some modification in the case of monitoring in the case of databases in the case of security or in the case of a capacity let's look into the capacity so here you can actually do the modification so in the capacity you can select the instance type also by default it is t2.micro but in case if your application requires a larger type of an instance then you can actually go for the instance type as well similarly you can define your emi ids also because obviously for the application to run you would require the operating system also so you can select that particular ami id for your operating system as well let's cancel that likewise you have many other features that you can actually define here from the dashboard and you don't have to go to the ec2 dashboard to do the modifications now let's go and create an environment let's assume that we are going with the default configuration so this is going to create our environment the environment is being created and you can get the environment and the logs defined in the dashboard itself so you see that the beanstalk environment is being initiated the environment is being started and in case if there would be any errors or if it is deployed correctly you will get all the logs here itself now the environments are basically color coded so there are different color codings that are defined if you get the environment in a green color that means everything is good to go so here you can see that it has created an elastic ip it has checked the health of the environment now it has created the security groups and that would be an auto security groups created by the bean stock and the environment creation has been started you can see that elastic bean stock has amazon s3 storage bucket for your environment data as well this is the url through which you will be accessing the environment but right now we cannot do that since the environment is being created let's click on the application name and here you can see that it is in a gray color that means right now the build is being done it is being created once it will be successfully created it should change to the green color and then we will be able to access our environment using the url now if i move to the ec2 instances and see in the ec2 dashboard if i see whether the instance is being created by the bean stock or not so let's see and let's see what are the differences in terms of creating an instance manually and getting it created from the bean stock so click on the easy to let's go to the old ec2 experience that's what we are familiar with and let's see what's there in the dashboard so here you can see one running instance let's open that and the xyz environment which was created from the beanstalk is being initiated the instance is being initiated and that is something which is being done by the mean stock itself we have not gone to the dashboard and created it manually now in the security groups if you see that here the aws mean stock security groups are defined it has the elastic ips also defined so everything is being created by the beanstalk itself right now let's go back to the beanstalk and let's look into the status of our environment whether the color coding has been changed from gray to green or not and here you can see the environment is successfully created and we have that environment colored in green we will access the environment and it says it's a web server environment its platform is node.js running on 64-bit amazon linux ami and it says samples app sample application health status is okay now the other thing is that if you do not want to use the web console the management console to access the main stock then the beanstalk offers you the elastic beanstalk cli as well so you can install the command line interface and then you have the command references cli command references that you can actually play with and get your applications deployed using the bean stock itself so this is one of the sample cli commands that you can actually look into now let's look into the environment let's click on the environment and we'll be represented with the url it says health is okay these are the logs that you have to follow in case if there are any issues the platform is nodejs that is what we selected now the next thing is you just have to upload and deploy your applications so just click on upload and deploy select the version label or the name select file and wherever your application is hosted at just select that upload it and deploy your application you'll see that the like your environment is created similarly your application will be deployed automatically on the instance and from this url you will be able to view the output it is as simple as just like you have to follow these four steps now let's see whether the node.js environment is running on our instance before deploying an application so we'll just click on this url since the beanstalk has already opened up the security groups or http port 80 for all we can actually view that output directly from the url so we have the node.js running that's visible here and after that you just have to upload and deploy your application and then from that url you can get the output now this url you can map it with the root 53 service so using the root 53 dns services the domain names can be pointed to the elastic bean stock url and from there it can be pointed to the applications that are running on the ec2 instance whether you wanted to point it to the url directly using the mean stock you can do that otherwise as we saw in the slides you can use the root 53 pointer to the load balancer and then point it to the instances directly also once it is created by the mean stock so that was the demo guys uh with respect to the bean stock and how we can actually run the environments apart from that the operational task like system operations you can manage all these things from the environment dashboard itself so you have the configurations you have the logs you can actually check the health status of your environment you can do the monitoring and you can actually get the alarms and the events here so let's say if i wanted to if i wanted to see the logs i can request for the logs here itself and i will be represented with the full log report and i can now download that log file and i can view the locks so it's in the so we have this bundle locks in the zip file all right so if you want to see some kind of uh logs with respect to elastic bean stock activity it's in the form of a notepad and here you can see what all configurations the beanstalk has done on your environment on your instance similarly you can go for the health monitoring alarms events and all those things if getting your learning started is half the battle what if you could do that for free visit skill up by simplylearn click on the link in the description to know more hi this is the fourth lesson of the aws solutions architect course migrating to the cloud doesn't mean that resources become completely separated from the local infrastructure in fact running applications in the cloud will be completely transparent to your end users aws offers a number of services to fully and seamlessly integrate your local resources with the cloud one such service is the amazon virtual private cloud this lesson talks about creating virtual networks that closely resemble the ones that operate in your own data centers but with the added benefit of being able to take full advantage of aws so let's get started in this lesson you'll learn all about virtual private clouds and understand their concept you'll know the difference between public private and elastic ip addresses you'll learn about what a public and private sudden there is and you'll understand what an internet gateway is and how it's used you'll learn what route tables are and when they are used you'll understand what a nat gateway is we'll take a look at security groups and their importance and we'll take a look at network acls and how they're used in amazon vpc we'll also review the amazon vpc best practices and also the costs associated with running a vpc in the amazon cloud welcome to the amazon virtual private cloud and subnet section in this section we're going to have an overview of what amazon vpc is and how you use it and we're also going to have a demonstration of how to create your own custom virtual private cloud we're going to look at ip addresses and the use of elastic ip addresses in aws and finally we'll take a look at subnets and there'll be a demonstration about how to create your own subnets in an amazon vpc and here are some other terms that are used in vpcs there's subnets root tables elastic ip addresses internet gateways nat gateways network acls and security groups and in the next sections we're going to take a look at each of these and build our own custom vpc that we'll use throughout this course amazon defines a vpc as a virtual private cloud that enables you to launch aws resources into a virtual network that you've defined this virtual network closely resembles a traditional network that you'd operate in your own data center but with the benefits of using the scalable infrastructure of aws a vpc is your own virtual network in the amazon cloud which is used as the network layer for your ec2 resources and this is a diagram of default vpc now there's a lot going on here so don't worry about that what we're going to do is break down each of the individual items in this default vpc over the coming lesson but what you need to know is that a vpc is a critical part of the exam and you need to know all the concepts and how it differs from your own networks throughout this lesson we're going to create our own vpc from scratch which you'll need to replicate at the end of this so you can do well in the exam each vpc that you create is logically isolated from other virtual networks in the aws cloud it's fully customizable you can select the ip address range create subnets configure route tables set up network gateways define security settings using security groups and network access control lists so each amazon account comes with a default vpc that's pre-configured for you to start using straight away so you can launch your ec2 instances without having to think about anything we mentioned in the opening section a vpc can span multiple availability zones in a region and here's a very basic diagram of a vpc it isn't this simple in reality and as we saw in the first section here's the default amazon vpc which looks kind of complicated but what we need to know at this stage is that cidr block for the default vpc is always a 16 subnet mask so in this example it's 172.31.0.0.16. what that means is this vpc will provide up to 65536 private ip addresses so in the coming sections we'll take a look at all of these different items that you can see on this default vpc but why wouldn't you just use the default vpc well the default vpc is great for launching new instances when you're testing aws but creating a custom vpc allows you to make things more secure and you can customize your virtual network as you can define your own ip address range you can create your own subnets that are both private and public and you can tighten down your security settings by default instances that you launch into a vpc can't communicate with your own network so you can connect your vpcs to your existing data center using something called hardware vpn access so that you can effectively extend your data center into the cloud and create a hybrid environment now to do this you need a virtual private gateway and this is the vpn concentrator on the amazon side of the vpn connection then on your side in your data center you need a customer gateway which is either a physical device or a software application that sits on your side of the vpn connection so when you create a vpn connection a vpn tunnel comes up when traffic is generated from your side of the connection vpc peering is an important concept to understand a parent connection can be made between your own bpcs or with a vpc in another aws account as long as it's in the same region so what that means is if you have instances in vpca they wouldn't be able to communicate with instances in vpc b or c unless you set up appearing connection peering is a one-to-one relationship a vpc can have multiple peering connections to other vpcs but and this is important transitive peering is not supported in other words vpca can connect to b and c in this diagram but c wouldn't be able to communicate with b unless they were directly paired also vpcs with overlapping cidrs cannot be paired so in this diagram you can see they all have different ip ranges which is fine but if they had the same ipo ranges they wouldn't be able to be paired and finally for this section if you delete the default vpc you have to contact aws support to get it back again so be careful with it and only delete it if you have good reason to do so and know what you're doing this is a demonstration of how to create a custom vpc so here we are back at the amazon web services management console and this time we're going to go down to the bottom left where the networking section is i'm going to click on the pc and the vpc dashboard will load up now there's a couple of ways you can create a custom vpc there's something called the vpc wizard which will build vpcs on your behalf from a selection of different configurations for example a vpc with a single public subnet or a bpc with public and private subnets now this is great because you click a button type in a few details and it does the work for you however you're not going to learn much or pass the exam if this is how you do it so we'll cancel that and we'll go to your vpcs and we'll click on create a vpc and we're presented with the create the vpc window so let's give our vpc a name i'm going to call that simply learn underscore vpc and this is the kind of naming convention i'll be using throughout this course next we need to give it the cidr block or the classless interdomain routing block so we're going to give it a very simple one 10.0.0 and then we need to give it the subnet mask so you're not allowed to go larger than 15. so if i try to put 15 in it says no not going to happen for a reference subnet mask of 15 would give you around 131 000 ip addresses and subnet 16 will give you 65 536 which is probably more than enough for what we're going to do next you get to choose the tenancy there's two options default and dedicated if you select dedicated then your ec2 instances will reside on hardware that's dedicated to you so your performance is going to be great but your cost is going to be significantly higher so i'm going to stick with default and we just click on yes create it'll take a couple of seconds and then in our vpc dashboard we can see our simply learn vpc has been created now if we go down to the bottom here to see the information about our new vpc we can see it has a root table associated with it which is our default route table so there it is and we can see that it's only allowing local traffic at the moment we go back to the vpc again we can see it's been given a default network acl and we'll click on that and have a look and you can see this is very similar to what we looked at in the lesson so it's allowing all traffic from all sources inbound and outbound now if we go to the subnet section and just widen the vpc area here you can see there's no subnets associated with the vpc we just created so that means we won't be able to launch any instances into our vpc and to prove it i'll just show you we'll go to the ec2 section so this is a glimpse into your future this is what we'll be looking at in the next lesson and we'll just quickly try and launch an instance we'll select any instance it doesn't matter any size not important so here the network section if i try and select simply learn vpc it's saying no subnets found this is not going to work so we basically need to create some subnets in our vpc and that is what we're going to look at in the next lesson now private ip addresses are ip addresses that are not reachable over the internet and they're used for communication between instances in the same network when you launch a new instance is given a private ip address and an internal dns host name that resolves to the private ip address of the instance but if you want to connect to this from the internet it's not going to work so then you'd need a public ip address which is reachable from the internet you can use public ip addresses for communication between your instances and the internet each instance that receives a public ip address is also given an external dns host name public ip addresses are associated with your instances from the amazon pool of public ip addresses when you stop or terminate your instance the public ip address is released and a new one is associated when the instance starts so if you want your instance to retain this public ip address you need to use something called an elastic ip address an elastic ip address is a static or persistent public ip address that's allocated to your account and can be associated to and from your instances as required an elastic ip address remains in your account until you choose to release it there is a charge associated with an elastic ip address if it's in your account but not actually allocated to an instance this is a demonstration of how to create an elastic ip address so we're back at the amazon web services management console we're going to head back down to the networking vpc section and we'll get to the vpc dashboard on the left hand side we'll click on elastic ips now you'll see a list of any elastic ips that you have associated in your account and remember any the elastic ip address that you're using that isn't allocated to something you'll be charged for so i have one available and that is allocated to an instance currently so we want to allocate a new address and it reminds you that there's a charge if you're not using it i'm saying yes allocate and it takes a couple of seconds and there's our new elastic ip address now we'll be using this ip address to associate with the nat gateway when we build that aws defines a subnet as a range of ip addresses in your bpc you can launch aws resources into a subnet that you select you can use a public subnet for resources that must be connected to the internet and a private subnet for resources that won't be connected to the internet the netmask for the default subnet in your vpc is always 20 which provides up to 4096 addresses per subnet and a few of them are reserved for aws use a vpc can span multiple availability zones but the subnet is always mapped to a single availability zone this is important to know so here's our basic diagram which we're now going to start adding to so we can see the virtual private cloud and you can see the availability zones and now inside each availability zone we've rated a subnet now you won't be able to launch any instances unless there are subnets in your vpc so it's good to spread them across availability zones for redundancy and failover purposes there's two different types of subnet public and private you use a public subnet for resources that must be connected to the internet for example web servers a public subnet is made public because the main route table sends the subnets traffic that is destined for the internet to the internet gateway and we'll touch on internet gateways next private subnets are for resources that don't need an internet connection or that you want to protect from the internet for example database instances so in this demonstration we're going to create some subnets a public and a private subnet and we're going to put them in our custom vpc in different availability zones so we'll head to networking and vpc wait for the vpc dashboard to load up we'll click on subnets we'll go to create subnet and i'm going to give the subnet a name so it's good to give them meaningful names so i'm going to call this first one for the public subnet 10.0.1.0 and i'm going to put this one in the us east 1b availability zone and i'm going to call that simplylearn public so it's quite a long name i understand but at least it makes it clear for what's going on in this example so we need to choose a vpc so we obviously want to put it in our simply learn vpc and i said i wanted to put it in us east 1b i'm using the north virginia region by the way so we click on that then we need to give it the cidr block now as i mentioned earlier when i typed in the name that's the range i want to use and then we need to give it the subnet mask and we're going to go with 24 which should give us 251 addresses in this range which obviously is going to be more than enough if i try and put a different value in that's unacceptable to amazon it's going to say it's going to give me an error and tell me not to do that let's go back to 24 and click i'm going to cut and paste this by the way just because i need to type something very similar for the next one click create and it takes a few seconds okay so there's our new subnet and i just widen this you can see so that's the ip range that's the availability zone it's for simply learn and it's public so now we want to create the private so put the name in i'm going to give the private the ip address block of that i'm going to put this one in us east 1c and it's going to be the private subnet obviously i want it to be in the same vpc better but it is the zone of us east 1c and we're going to give it 10.0.2.0 24 and we'll click yes create again it takes a few seconds okay so we sort by name so there we are we can see now we've got our private subnet and our public subnet let me just type in simply then there we are so now you can see them both there and you can see they're both in the same vpc simply learn vpc now if we go down to the bottom you can see the root table associated with these vpcs and you can see that they can communicate with each other internally but there's no internet access so that's what we need to do next in the next lesson you're going to learn about internet gateways and how we can make these subnets have internet access welcome to the networking section in this section we're going to take a look at internet gateways root tables and nat devices and we'll have a demonstration on how to create each of these aws vpc items so to allow your vpc the ability to connect to the internet you need to attach an internet gateway and you can only attach one internet gateway per vpc so attaching an internet gateway is the first stage in permitting internet access to instances in your vpc now here's our diagram again and now we've added the internet gateway which is providing the connection to the internet to your vpc but before you can configure internet correctly there's a couple more steps for an ec2 instance to be internet connected you have to adhere to the following rules firstly you have to attach an internet gateway to your vpc which we just discussed then you need to ensure that your instances have public ip addresses or elastic ip addresses so they're able to connect to the internet then you need to ensure that your subnets root table points to the internet gateway and you need to ensure that your network access control and security group rules allow relevant traffic to flow to and from your instance so you need to allow the rules to let in the traffic you want for example http traffic after the demonstration for this section we're going to look at how root tables access control lists and security groups are used in this demonstration we're going to create an internet gateway and attach it to our custom vpc so let's go to networking bpc bring up the vpc dashboard and on the left hand side we click on internet gateways so here's a couple of internet gateways i have already but i need to create a new one so create internet gateway i'll give it a name which is going to be simply learn internet gateway igw and i'm going to click create so this is an internet gateway which will connect a vpc to the internet because at the moment our custom vpc has no internet access so there it's created simply then i gw but it's state is detached because it's not attached to anything so let me try and attach it to a vpc and it gives me an option of all the vpcs that have no internet gateway attached to them currently so i only have one which is simply then bpc yes attach now you can see our vpc has internet attached and you can see that down here so let's click on that and it will take us to our vpc but before any instances in our vpc can access the internet we need to ensure that our subnet root table points to the internet gateway and we don't want to change the main route table we want to create a custom route table and that's what you're going to learn about next a route table determines where network traffic is directed it does this by defining a set of rules every subnet has to be associated with a root table and a subnet can only be associated with one root table however multiple subnets can be associated with the same route table every vpc has a default route table and it's good practice to leave this in its original state and create a new route table to customize the network traffic routes associated with your vpc so here's our example and we've added two route tables the main route table and the custom route table the new route table or the custom route table will tell the internet gateway to direct internet traffic to the public subnet but the private subnet is still associated to the default route table the main route table which does not allow internet traffic to it all traffic inside the private subnet is just remaining local in this demonstration we're going to create a custom route table associated with our internet gateway and associate our public subnet with it so let's go to networking and vpc the dashboard will load and we're going to go to route tables now our vpc only has its main route table at the moment the default one it was given at the time it was created so we want to create a new route table and we want to give it a name so we're going to call it simply learn i'm going to call it root table rtb for sure and then we get to pick which vpc we want to put it in so obviously we want to use simply learn vpc so we click create which will take a couple of seconds and here you are here's our new route table so what we need to do now is change its route so that it points to the internet gateway so if we go down here to roots at a minute you can see it's just like our main route table it just has local access so we want to click on edit and we want to add another route so the destination is the internet which is all the zeros and our target and we click on this it gives us the option of our internet gateway which we want to do so now we have internet access to this subnet sorry to this route table and we click on save save was successful so now we can see that as well as local access we have internet access now at the moment if we click on subnet associations you do not have any subnet associations so basically both both our subnets the public and private subnets are associated with the main route table which doesn't have internet access so we want to change this we'll click on edit and we want our public subnet to be associated with this root table so click on save so it's just saving that so now we can see that our public subnet is associated with this route table and this route table is associated with the internet gateway so now anything we launch into the public subnet will have internet access but what if we wanted our instances in the private subnet to have internet access well there's a way of doing that with a nat device and that's what we're going to look at in the next lecture you can use a nat device to enable instances in a private subnet to connect to the internet or other aws services but prevent the internet from initiating connections with the instances in the private subnet so we talked earlier about public and private subnets to protect your assets from being directly connected to the internet for example your web server would sit in the public subnet in your database in the private subnet which has no internet connectivity however your private subnet database instance might still need internet access or the ability to connect to other aws resources if so you can use a network address translation device or a nat device to do this and that device forwards traffic from your private subnet to the internet or other aws services and then sends the response back to the instances when traffic goes to the internet the source ip address of your instance is replaced with the nat device address and when the internet traffic comes back again then that device translates the address to your instance's private ip address so here's our diagram which is getting ever more complicated and if you look in the public subnet you can see we've now added and that device and you have to put that devices in the public subnet so that they get internet connectivity aws provides two kinds of nat devices and that gateway and a nat instance aws recommends a nat gateway as it's a managed service that provides better availability and bandwidth than that instances each nat gateway is created in a specific availability zone and is implemented with redundancy in that zone and that instance is launched from a nat ami and amazon machine image and runs as an instance in your vpc so it's something else you have to look after whereas in that gateway being a fully managed service means once it's installed you can pretty much forget about it and that gateway must be launched into a public subnet because it needs internet connectivity it also needs an elastic ip address which you can select at the time of launch once created you need to update the route table associated with your private subnet to point internet bound traffic to the nat gateway this way the instances in your private subnets can communicate with the internet so if you remember back to the diagram when we had the custom route table which was pointed to the internet gateway now we're pointing our main route table to the nat gateway so the private subnet also gets internet access but in a more secure manner welcome to the create a nat gateway demonstration where we're going to create a nat gateway so that the instances in our private subnet can get internet access so we'll start by going to networking and vpc and the first thing we're going to do is take a look at our subnets and you'll see why shortly so here are our simply learned subnets so this is the private subnet that we want to give internet access but if you remember from the section that gateways need to be placed in public subnets so i'm just going to copy the name of this subnet id for the public subnet and you'll see why in a moment so then we go to nat gateways on the left hand side and we want to create a new nat gateway so we have to put a subnet in there so we want to choose our public subnet as you can see it truncates a lot of the subnet names on this option so it's a bit confusing so we know that we want to put it in our simply learn vpc in the public subnet but you can see it's truncated so it's actually this one at the bottom but what i'm going to do is just paste in the subnet id which i copied earlier so there's no confusion then we need to give it an elastic ip address now if you remember from the earlier demonstration we created one so let's select that but if you hadn't allocated one you could click on the create new eip button so we'll do that okay so it's telling me my nat gateway has been created and in order to use your netgateway ensure that you edit your route table to include a route with a target of and then on that gateway id so it's given us the option to click on our edit route tables so we'll go straight there now here's our here's our route tables now here's the custom route table that we created earlier and this is the default the main route over which was created when we launched our when we created our vpc so we should probably give this a name so that we know what it is so let me just call this simplylearn rtb main so now we know that's our main route table so if you take a look at the main route table and the subnet associations you can see that our private subnet is associated with this table so what we need to do is put a route in here that points to the nat gateway so if we click on routes and edit and we want to add another route and we want to say that all traffic can either go to the simply then internet gateway which we don't want to do we want to point it to our nat instance which is this nat id here and we click save so now any instances launched in our private subnet will be able to get internet access via on that gateway welcome to the using security groups and network acl section in this section we're going to take a look at security groups and network acls and we're going to have a demonstration on how we create both of these items in the amazon web services console a security group acts as a virtual firewall that controls the traffic for one or more instances you add rules to each security group that adow traffic to or from its associated instances basically a security group controls the inbound and outbound traffic for one or more ec2 instances security groups can be found on both the ec2 and vpc dashboards in the aws web management console we're going to cover them here in this section and you'll see them crop up again in the ec2 lesson and here is our diagram and you can see we've now added security groups to it and you can see that ec2 instances are sitting inside the security groups and the security groups will control what traffic flows in and out so let's take a look at some examples and we'll start with a security group for a web server now obviously a web server needs http and https traffic as a minimum to be able to access it so here is an example of the security group table and you can see we're allowing http and https the ports that are associated with those two and the sources and we're allowing it from the internet we're basically allowing all traffic to those ports and that means any other traffic that comes in on different ports would be unable to reach the security group and the instances inside it let's take a look at an example for a database server security group now imagine you have a sql server database then you would need to open up the sql server port so that people can access it and which is port 1433 by default so we've added that to the table and we've allowed the source to come from the internet now because it's a windows machine you might want rdp access so you can log on and do some administration so we've also added rdp access to the security group now you could leave it open to the internet but that would mean anyone could try and hack their way into your box so in this example we've added a source ip address of 10.0.0.0 so only ipa ranges from that address can rdp to the instance now there's a few rules associated with security groups by default security groups allow all outbound traffic so if you want to tighten that down you can do so in a similar way to you can define the inbound traffic security group rules are always permissive you can't create rules that deny access so you're allowing access rather than denying it security groups are stateful so if you send a request from your instance the response traffic for that request is allowed to flow in regardless of the inbound security group rules and you can modify the rules of a security group at any time and the rules are applied immediately welcome to the create security group demonstration where we're going to create two security groups one the hostdb servers and one the host web servers now if you remember from the best practices section it said it was always a good idea to tear your applications into security groups and that's exactly what we're going to do so we go to networking and vpc to bring up the vpc dashboard on the left hand side under security we click on security groups now you can also get to security groups from the ec2 dashboard as well so here's a list of my existing security groups but we want to create a new security group and we're going to call it simply learn web server sg security group and we'll give the group name as the same and our description is going to be simply learn web servers security groups okay and then we need to select our vpc now it defaults to the default vpc but obviously we want to put it in our simply learn vpc so we click yes create takes a couple of seconds and there it is there's our new security group now if we go down to the rules the inbound rules you can see there are none so by default a new security group has no inbound rules but what about outbound rules if you remember from the lesson a new security group by default allows all traffic to be outbound and there you are all traffic has destination of everywhere so all traffic is allowed we're going to add some rules so let's click on inbound rules click on edit now this is going to be a web server so if we click on the drop down we need to give it http so you can either choose custom tcp rule and type in your own port ranges or you can just use the ones they have for you so http this pre-populates the port range and then here you can add the source now if i click on it it's giving me the option of saying allow access from different security groups so you could create a security group and say i only accept traffic from a different security group which is a nice way of securing things down you could also put in here just your ip address so that only you could do http requests to the instance but because it's a web server we want people to be able to see our website otherwise it's not going to be much use so we're going to say all traffic so all source traffic can access our instance on port http 80. i want to add another rule because we also want to do https which is hiding from me there we are and again we want to do the same and also because this is going to be a linux instance we want to be able to connect to the linux instance to do some work and configuration so we need to give it ssh access and again it would be good practice to tie it down to your specific ip or an ip range but we're just going to do all for now and then we click on save and there we are there we have our ranges so now we want to create our security group for our db servers so let's click create security group and then we'll go through and give it a similar name simply learn db servers sg and the description is going to be simply learn db servers security group and our vpc is obviously going to be simply learn vpc so let's click yes create wait a few seconds and here's our new security group as you can see it has no inbound rules by default and outbound rules allow all traffic so this is going to be a sql server database server and so we need to allow sql server traffic into the instance so we need to give it microsoft sql port access now the default port for microsoft sql server is 1433. now in reality i'd probably change the port the sql server was running on to make it more secure but we'll go with this for now and then the source so we could choose the ip ranges again but what we want to do is place the db server in the private subnet and allow the traffic to come from the web server so the web server will accept traffic and the web server will then go to the database to get the information it needs to display on its web on the website or if people are entering information into the website we want the information to be stored in our db server so basically we want to say that this the db servers can only accept sql server traffic from the web server security group so we can select the simply then web server security group as the source traffic for microsoft sql server data so we'll select that now our sql server is obviously going to be a windows instance so from time to time we might not we might need to log in and configure it so we want to give rdp access now again you would probably put a specific ip range in there we're just going to do all traffic for now then we click save and there we are so now we have two security groups db servers and web servers a network acl is a network access control list and it's an optional layer of security for your vpc that acts as a firewall for controlling traffic in and out of one or more of your subnets you might set up network acls with rules similar to your security groups in order to add an additional layer of security to your vpc here is our network diagram and we've added network acls to the mix now you can see they sit somewhere between the root tables and the subnets this diagram makes it a little bit clearer and you can see that a network acl sits in between a root table and a subnet and also you can see an example of the default network acl which is configured to allow all traffic to flow in and out of the subnets to which is associated each network acl includes a whose rule number is an asterisk this rule ensures that if a packet doesn't match any of the other numbered rules it's denied you can't modify or remove this rule so if you take a look at this table you can see on the inbound some traffic would come in and it would look for the first rule which is 100 and that's saying i'm allowing all traffic from all sources so that's fine the traffic comes in if that rule 100 wasn't there it would go to the asterix rule and the aztecs rule is saying traffic from all sources is denied let's take a look at the network acl rules each subnet in your vpc must be associated with an acl if you don't assign it to a custom acl it will automatically be associated to your default acl a subnet can only be associated with one acl however an acl can be associated with multiple subnets an acl contains a list of numbered rules which are evaluated in order starting with the lowest as soon as a rule matches traffic it's applied regardless of any higher numbered rules that may contradict it aws recommends incrementing your rules by a factor of 100 so there's plenty of room to implement new rules at a later date unlike security groups acls are stateless responses to allowed inbound traffic are subject to the rules for outbound traffic welcome to the network acl demonstration where we're just going to have an overview of ocls where they are in the dashboard now you don't need to know a huge amount about them for the exam you just need to know how they work and where they are so let's go to networking and vpc and on when the dashboard loads on the left hand side under security there's network acls so let's click on that now you can see some acls that are in my my aws account so we want the one that's associated with our simply learn vpc so if we extend this vpc column that's our network acl simply then vpc now let's give it a name because it's not very clear to see otherwise also i'm kind of an obsessive tagger so let's call it simply learn acl and click on the tick so there we are so now it's much easier to see so we click on inbound rules so this is exactly what we showed you in the lesson the rule is 100 so that's the first rule that's going to get evaluated and it's saying allow all traffic from all sources and the outbound rules are the same so if you wanted to tighten down the new rule you could click edit we would give it a new rule number say which would be 200 so you should always increment them in 100 so that means if you had 99 more rules you needed to put in place you'd have space to put them in in between these two and then you could do whatever you wanted you could say you know we are allowing http access from all traffic and we're allowing or you could say actually you know what we're going to deny it so this is the way of blacklisting traffic into your vpc now i'm not going to save that because we don't need it but this is where network acls sit and this is where you would make any changes it's also worth having a look at the subnet associations with your acl so we have two subnets in our simply learn vpc so we would expect to see both of them associated with this network acl because it's the default and there they are this both are public and our private subnets are associated and you can also see up here on the on the dashboard it says default so this is telling us this is our default acl if you did want to create a new network acl you would click create network acl you'd give it a name just say new acl and then you would associate it with your vpc so we would say simply learn vpc takes a few seconds and there we are there we have our new one now you can see this one says default no because it obviously isn't the default acl for our simply learn vpc and it has no subnets associated with it so let's just delete that because we don't need it but there you are there's a very brief overview of network pcls welcome to the amazon vpc best practices and costs where we're going to take a look at the best practices and the costs associated with the amazon virtual private cloud always use public and private subnets you should use private subnets to secure resources that don't need to be available to the internet such as database services to provide secure internet access to the instances that reside in your private subnets you should provide a nat device when using that devices you should use a nat gateway over nat instances because they're a managed service and require less administration effort you should choose your cidr blocks carefully amazon vpc can contain from 16 to 65 536 ip addresses so you should choose your cidr block according to how many instances you think you'll need you should also create separate amazon vpcs for development staging test and production or create one amazon vpc with separate subnets with a subnet each for production development staging and test you should understand the amazon vpc limits there are various limitations on the vpc components for example you're allowed 5 vpcs per region 200 subnets per vpc 200 route tables per vpc 500 security groups per vpc 50 in and outbound rules per vpc however some of these rules can be increased by raising a ticket with aws support you should use security groups and network acls to secure the traffic coming in and out of your vpc amazon advises to use security groups for white listing traffic and network acls for blacklisting traffic amazon recommends tiering your security groups you should create different security groups for different tiers of your infrastructure architecture inside vpc if you have web tiers and db tiers you should create different security groups for each of them creating tier wise security groups will increase the infrastructure security inside the amazon vpc so if you launch all your web servers in the web server security group that means they'll automatically all have http and https open conversely the database security group will have sql server ports already open you should also standardize your security group naming conventions following a security group naming convention allows amazon vpc operation and management for large-scale deployments to become much easier always span your amazon vpc across multiple subnets in multiple availability zones inside a region this helps in architecting high availability inside your vpc if you choose to create a hardware vpn connection to your vpc using virtual private gateway you are charged for each vpn connection hour that your vpn connection is provisioned and available each partial vpn connection hour consumed is billed as a full hour you'll also incur standard aws data transfer charges for all data transferred via the vpn connection if you choose to create a nat gateway in your vpc you are charged for each nat gateway hour that your nat gateway is provisioned and available data processing charges apply for each gigabyte processed through the net gateway each partial nat gateway hour consumed is billed as a full hour this is the practice assignment for designing a custom vpc where you'll create a custom bpc using the concepts learnt in this lesson using the concepts learnt in this lesson recreate the custom vpc as shown in the demonstrations the vpc name should be simply learn vpc the cidr block should be 10.0.0.0.16 there should be two subnets one public with a range of 10.0.1.0 and one private with a range of 10.0.2.0 and they should be placed in separate availability zones there should be one internet gateway and one that gateway and also one custom route table for the public subnet also create two security groups simply learn web server security group and simply learn db server security group so let's review the key takeaways from this lesson amazon virtual private cloud or vpc enables you to launch aws resources into a virtual network that you've defined this virtual network closely resembles a traditional network that you'd operate in your own data center but with the benefits of using scalable infrastructure of aws there are three types of ip address in aws a private ip address this is an ip address that's not reachable over the internet and it's used for communication between instances in the same network a public ip address is reachable from the internet which you can use for communication between your instances and the internet and there's an elastic i p address this is a static public persistent ip address that persists after an instance restarts whereas a public ip address is raised associated after each restart amazon defines a subnet as a range of ip addresses in your vpc you can launch aws resources into a subnet that you select and a subnet is always mapped to a single availability zone use a public subnet for resources that must be connected to the internet and a private subnet for resources that won't be connected to the internet to allow your vpc the ability to connect to the internet you need to attach an internet gateway to it and you can only attach one internet gateway per vpc a root table determines where network traffic is directed it does this by defining a set of rules every subnet has to be associated with a root table and a subnet can only have an association with one root table however multiple subnets can be associated to the same root table and you can use a nat device to enable instances in a private subnet to connect to the internet or other aws services but and that device will prevent the internet from initiating connections with instances inside your private subnet a security group acts as a virtual firewall that controls the traffic for one or more instances you add rules to each security group that allow traffic to or from its associated instances a network access control list or network acl is an optional layer of security for your vpc that acts as a firewall for controlling traffic in and out of one or more of your subnets today's session is on aws sagemaker let's look into what we have in our today's session so what's in it for you we would be covering what is aws why do we need aws sage maker what is aws sage maker services what are the benefits of using the aws sagemaker machine learning with aws sagemaker how to train a model with aws stage how to validate a model with aws and the companies that are using aws shmaker along with that we will be covering up one live demo on the aws platform now let's understand what is aws so what is aws it's an amazon web services it's a largest or most widely used public cloud platform offered by amazon it provides services over the internet aws services can be used to build monitor and deploy any type of application in the cloud aws also uses the subscription pricing model that means you only pay for whatever the services you use for now why do we need aws sagemaker let's look into it so let's consider an example of one of the company that is proquest now before aws age maker the proquest is a global information content and technology company that provides valuable content such as ebooks newspapers etc to the users before awschmaker the proquest requirement was to have a better user experience maximum relevant search results now after aws shmaker they were able to achieve those results so they achieved more appealing video user experience they achieved more relevant search results for the users now what do we mean by aws sagemaker why this service is primarily used so amazon sage maker is a cloud machine learning platform that helps users in building training tuning and deploying machine learning models in a production ready hosted environment so it's kind of a machine learning service which is already hosted on the aws platform now what are the benefits of using aws sagemaker uh the key benefits of using aw sagemaker are it reduces machine learning data cost so you can do the cost optimization while running this particular service on the aws all ml components are stored in a particular place in a dashboard so they can be managed together highly scalable so it can be scalable on you can scale this particular service on the fly it trains the models quite faster maintains the uptime so you can be assured that your workloads will be running all the time it will be available all the time high data security so security becomes a major concern on the cloud platforms and it ensures that you have the high data security along with that you can do a data transfer to different aws services like s3 bucket and all with these simple data transfer techniques now machine learning with aws sagemaker let us look into it so machine learning with aws sagemaker is a three-step function so one is to build second is to test and tune the model and third is to deploy the model now with the build it provides more than 15 widely used ml algorithms for training purpose now to build a model you can collect and prepare training data or you can select from the amazon s3 bucket also choose and optimize the required algorithm so some of the algorithms that you can select are k-means linear regressions logistic regression sage maker helps developers to customize ml instances with the jupiter notebook interface in the test and tune you have to set up and manage the environment for training so you would need some sample data to train the model so train and tune a model with the amazon sage maker sage maker implements hyper parameter tuning by adding a suitable combination of algorithm parameters also it divides the training data and stores that in the amazon s3 s3 is a simple storage service which is primarily used for storing the objects and the data hence it is used for storing and recovering data over the internet and below you can see that aws sagemaker uses amazon s3 to store data as it's safe and secure also it divides the training data and stores in amazon s3 where the training algorithm code is stored in the ecr ecr stands for elastic container registry which is primarily used for containers and dockers ecr helps users to save monitor and deploy docker and the containers later sage maker sets up a cluster for the input data trains it and stores it in the amazon s3 itself so this is done by the sagemaker itself after that you need to deploy it so suppose you want to predict limited data at a time you use amazon sage maker hosting services for that okay but if you want to get prediction for an entire data set prefer using amazon sage maker batch transform now the last step that is to deploy the model so once tuning is done models can be deployed to sagemaker endpoints and in the end point real time prediction is performed so you would have some data which you would reserve and validate your model whether it is working correctly or not now evaluate your model and determine whether you have achieved your business goals now the other aspect is how we can train a model with aws stage maker so this is basically a flow diagram which shows you how to train a model with the aws sagemaker and here we have used couple of services of an aws to get that done so model training in age maker is done on machine learning compute instances and here we can see there are two machine learning compute instances used as helper code and the training code along with that we are using 2s3 buckets and the ecr for the container registry now let's look into what are the ways to train the model as per the slides so below are the following requirements to train a model so here in the diagram you can see these are the following requirements to train a model the url of an amazon s3 bucket where the training data is stored that is mentioned the compute resources on machine learning compute instances so these are all your machine learning compute instances then the url of an amazon s3 bucket where the output will be stored and the path of aws elastic container registry where the code data is safe the inference code image lies in the elastic container registry now what are these calls these are called as the training jobs now when a user trains a model in amazon sage maker he she creates a training job so we need to first create a training job and then the input data is fetched from the specified amazon s3 bucket once the training job is built amazon sage maker launches the ml compute instances so these compute instances will be launched once the training job is built then it trains the model with the training code and the data set and it shows the output and model rt graphs in the aws s3 bucket so this is done automatically now here the helper code performs a task when the training code fails the interference code which is in the elastic container registry consists of multiple linear sequence containers that process the request for inference on data the ec2 container registry is a container registry that helps users to save monitor and deploy container images whereas container images are the ready applications once the data is trained the output is stored in the specified amazon s3 bucket so here you can see the output will be stored here to prevent your algorithm being deleted save the data in amazon's sagemaker critical system which can process you on your ml compute instances now how to validate a model let's look into it so you can evaluate your model using offline or using the historical data so first thing is that you can do the offline testing to validate a model you can do an online testing with the live data so if you have a live data coming or real time streams coming you can validate a model from there itself you can validate using a holdout set and also you can validate using the k fold validation now use historical data to send requests to the model through the jupiter notebook in amazon sagemaker for the evaluation online testing with livedata deploys multiple models into the endpoints of amazon sagemaker and directs live traffic to the model for validation validating using a holdout set is part of the data is set aside where which is called holdout set so the part of the data is left which is basically called as the holdout set this data is not used for the model training so later when the model is trained with the remaining input data and generalize the data based on what is learnt initially so whatever the data which is left out will be used for validating a model because we have not used the data while training a model the k-fold validation is the input data is split into two parts one part is called k which is the validation data for testing the model and the other part is k minus 1 which is used as a training data now based on the input data the machine learning model evaluates the final output now the companies that are using aws sagemaker one is the adp so you must be knowing about adb zlando dow jones which is the stock market proquest and the intuit now let's look into the demo that how we can actually run the aws sagemaker so we'll use the r algorithm and then package the algorithm as a container for building training and deploying a model we are going to use the jupyter notebook for that for model building for model training for model deployment and the code for the demo is in the below link so you can see here that from this link you can get the code for the demo let's try to do a demo on the aws now i would be using a link which is provided by amazon to build train and deploy the machine learning model on the sagemaker as you can see on my screen and in this tutorial you would have some steps where you can put those steps and the code python codes into your aws sagemaker jupiter lab so in this tutorial you will learn how to use amazon sagemaker to build train and deploy a machine learning model and for that we will use the popular xt boost ml algorithm for this exercise so first of all what you need to do is you have to go to the aws console and there you have to create a notebook instance so in this tutorial you will be creating a notebook instance you will prepare the data train the model to learn from the data deploy the model evaluate your ml model's performance and once all those activities are done then we will see how we can actually remove all the resources in order to prevent the extra costing now the first step is we have to enter to the amazon stage maker console so here you can see i'm already logged in into the sagemaker console you can click on the services search for the sagemaker here and here you get the amazon sagemaker service now the next step is that we have to create a notebook instance so we will select the notebook instance from the sagemaker service and then after the notebook instance is selected we'll put a name to our instance and we'll create a new im role for that so let's wait for the sage maker studio to open so here you can see the studio is open and you just have to click on the notebook instances and here you have to create a notebook instance so here you can see couple of notebook instances have already been created one of them is in service so this is the notebook instance that we are going to use for creating the demo model i'll show you how you can create a notebook instance you just have to click on create notebook instance button and put your notebook instance name so you can put something like demo dash sage maker 987 or we can put it as model we'll go with notebook instance type as default which is mlt2.medium and in the permission and encryptions under the i am role we'll click on create a new i am role now why we are creating a new im rule so that we can allow the sage maker to access any s3 bucket that has been created on our account just click on create a role and here you would see that the new im role will be created with the set of permissions the rest of the things will keep it as default and then you just have to click on create a notebook instance the notebook instance creation takes some time so you just have to wait for a couple of minutes to get that in service we already have one of the notebook instance that has been created so we will be using that to create a demo now going back to the steps so these are the steps that we have already performed now once the notebook instance is created then we have to prepare the data so in this step we will be using the sagemaker notebook instance to pre-process the data that would require to train the machine learning model and for that we would be opening up the jupiter notebook and then we have to select an environment a kernel environment in the jupiter notebook that would be conda underscore python 3. so let's follow these steps go back to the sage maker click on the notebook instances select the running notebook instance and here you would select the open jupiter lab now here you would see that the sagemaker would try to open up the jupiter notepad and we would be performing all our inputs into that jupiter notebook and executing the results there itself so just wait for the notebook to open now here you can see the jupyter lab notebook has been open so i would be selecting one of the notebook that has been created so this one so likewise you can create your own notebook also how you can do that first of all let me select the kernel environment so i would be selecting underscore python 3 and just click on select so how you can create your own notebook just have to click on file click on new and here you can select the notebook just name your notebook select the environment conda underscore python 3 to run this demo so i have my notebook open so in the tabs i would be putting up the python codes and i would be executing those codes to get the output directly so the next step is to prepare the data train the ml model and deploy it we will need to import some libraries and define a few environment variables in the jupyter notebook environment so i would be copying this code which you can see that would try to import numpy pandas these are all required to run the python syntax so just copy this code and paste it into your notebook right so once you do that execute your code and here you can see that you get the output which is that it has imported all the necessary libraries that have been defined in the code now the next step is we would create an s3 bucket into the s3 service and for that you have to copy this python code just that you have to edit it so you have to specify this bucket name that you want to get created so here i would provide the bucket name which should be unique should not overlap so something like sage maker dash demo is the name that i have selected for the bucket and now you have to execute that code it says that the s3 bucket has been created successfully with the name sage maker dash demo 9876 so this is something which you can verify so you can go to the s3 service and there you can verify whether the bucket has been created or not now the next task is that we need to download the data to the aws sagemaker instance and load it into the data frame and for that we have to follow this url so from this url which is build train deploy machine learning model would have a data in the form of bank underscore clean.csv and this will be deployed onto our sagemaker instance we'll copy this code and paste it here and execute the code so it says that it has successfully downloaded bank underscore clean.csv which is which has the data inside it and that has been loaded into the sage maker data frame successfully now we have a data to build and train our machine learning model so what we are going to do we are going to shuffle the data and we are going to split it one into the training data set and the other one into test data set so for the training data set we are going to use 70 of the customers that are listed in the csv file and 30 of the customers in the csv file data we will be using it as a test data to train the model so we will copy the following code into a new code cell and then we are going to run that code cell so i'll just copy it for training the data so that we can segregate the data model 70 for building the model and 30 for testing the data so click on run the execution and here you can see that we got the output successfully now we have to train the model from that data so how we are going to train that model and for that we will use sagemaker prebuilt xgboost model which is an algorithm so you will need to reformat the header and first column of the training data and load the data from the s3 bucket so what i'll do is i'll copy this syntax paste it in the note cell so it has the trained data it would train the model click on run execution now it is changing the s3 input class which will be renamed to training input because now we are training the model with the training data so we just have to wait for some time till it gets executed completely now the next thing is that we need to set up the amazon sage maker session to create an instance of the xgboost model so here we are going to create this sagemaker session say we are going to create an instance of the xt boost model which is an estimator so just copy that copy that code and paste it here execute it and here you can see that it will start it has basically changed the parameter image name to the image underscore uri in this sage maker python sdk v2 now we'll follow the next step that is with the data loaded in the xt boost estimator we'll set up train the model using gradient optimization and we'll copy the following code and that would actually start the training of the model so copy this code and this would actually start training the model using our input data that we have reserved 70 of that data that we have reserved for training the model so just copy that again initiate the execution and it will start the training job now we'll deploy the model and for that i would copy the deploy code put that in the cell and execute it so it says parameter image will be renamed to image uri and using already existing model so xd boost was deployed already uh if you have not done that if you're doing it first time so it will initiate another xd boost instance so where you can find your xt boost endpoints created you just have to scroll down and here under the interference click on the endpoints and you should find the xt boost endpoints defined here so here you can see that today i have created one xt boost endpoint and that is uh now in process of creating so just refresh it so it is still created it is going to take some time to get that in service now our endpoint is in service state so now we can use it so going forward with the next steps we'll try to predict whether the customer in the test data enroll for the bank product or not for that we are going to copy this code put that in the jupiter cell function and execute it so here it gives you the output that it has actually evaluated and the same output we got in the screenshot of the demo as well now we are going to evaluate the model performance so what we are going to do we are going to get the prediction done so based on the prediction we can conclude that you predicted a customer that will enroll for a certificate of deposit accurately for 90 of the customers in the test data with the precision of 65 percent for enrolled and 90 which are which haven't enrolled for it so for that we are going to copy this code and executed here in the cell so if it is predicted correctly that means our model is working absolutely fine so here you can say the overall classification rate is 89.5 percent and there is the accurate prediction that has been made by the model and that's what the output we can see here in the screenshot of a model so that means our model is absolutely working fine it has been built deployed and trained correctly now the next thing is that once you are done with that you terminate your resources and for that you just have to copy this code and put that in the cell function so that the additional resources and the endpoints and the buckets that have been created by the jupiter notepad should be terminated so that you would not be incurred with the extra costing so just execute it and here you would see that it has tried to it would try to terminate all the additional resources that we have created from the jupyter notebook today's tutorial is on aws cloudfront let's look into what we have today in the cloud front so what's in it for you we would be covering up the concept of what is aws what was earlier uh before aws cloudfront after aws cloudfront what were the services that were introduced how it benefited what do we mean by aws cloudfront benefits of the using aws cloudfront and how aws cloudfront actually is known as a content delivery service the name of the companies that are using aws cloud front and we would be covering up on live demo now aws is the amazon web services it's a cloud service provider that basically offers a multiple services variety of services such as compute power database storage networking and other resources so that you can create your solutions on the cloud and help the business grow now with aws you only pay for whatever the services you use so for example if you are using a service for a couple of hours then you pay for only that many hours that you have used that service before aws cloud front so there is an example that we are going to talk and that is you must be aware of an application called spotify so when you used to access spotify and when you click on it it kept on loading and at the end you used to get the error and the error was that the connection failed and why you received that error because of a latency issue probably a network error right so how you can solve these kind of a latency issues and that is also going to these kind of an issues are also going to impact the performance of an application so with the introduction of aws cloud from this problem of loading the application got resolved so after aws cloud front with the help of idle play slot front spotify gives the facility of updating new features access to million songs that you can access instantly so with the use of aws cloudfront or the latency issues were solved and successfully you can basically access your application now what we mean by aws cloudfront so aws cloudfront is a globally distributed network that is offered by aws amazon web services which securely delivers content to the end users across any geography with a higher transfer speed and an improve or a low latency now what are the benefits of aws cloudfront there are multiple benefits one is the cost effective so it helps you to do the cost optimization when you use the cloud front it is time saving so it is implemented easily and also a lot of issues with respect to accessing the application with respect to latency and all can be resolved content privacy so the content is placed to the end users and also to the cloudfront servers in a secured manner in a secured way it is highly programmable and you can make the changes amend the changes on the fly and you can target any location any geography across the globe along with that it helps you to get the content deliver quickly now how aws cloudfront delivers the content let's look into the architecture so this is a flow and the flow is with respect to how the user is going to get a content from the cloud front now the client first access a website by typing a url on the browser and in the step one it takes tries to access the application then the client requests when the website is open the client request for an object to download such as for example a particular file now at that time the dns routes user request to download that file to aws cloudfront the aws cloud front connects to the nearest edge locations edge locations are basically the servers where it caches the files documents and the web codes aws cloud front connects to its nearest edge location in order to serve the user the request at edge location aws cloud front looks for its requested cache file once the file is found let's say if the file is available in the cache of an edge location aws cloudfront then sends the file to the user otherwise if the file is not found in the cache memory of an edge location ews cloud front compares the requirement with the specification and share it with a respected server that means a web server or a server where the file is actually available the server the web server responds to the edge location by sending the file back to the cloudfront edge location and then as soon as the aws cloud front receives the file it shares with the client also adds the file to the cache of an edge location for a future reference this is how the flow of a cloud front is now the name of the companies that are using the aws cloud front so one of them is jio7 app which is which is a very popular app so it uses amazon cloudfront to deliver 15 petabytes of audio and video to its subscribers globally which is a huge data sky news it uses the service in order to unify the content for faster distribution to scrub subscribers or the discovery communications also uses the cloud front it uses the service for delivering api static asset and also the dynamic content and then the tv one eu streaming europe is basically also uses the cloudfront service that helps in improving latency and performance that results in fastest delivery of content now let's look into the demo how to use cloudfront to serve private s3 bucket as a website now i'm going to run a cloudfront distribution demo on the aws console and we'll basically try to deliver the content from a private s3 bucket and then map that with the domain name using the root 53 service so what we need for this demo we would need a domain url we would need a route 53 service we would need cloud front uh we have to create a cloud front distribution and that will be linked with our s3 bucket the private s3 bucket right and in the s3 bucket we would have one html file the index.html file so let's move into aws console so right now i have opened up the cloudfront distribution and here you can see that couple of distributions have already been created so what i'm going to do i'm going to create a distribution now there are two types of delivery method for your content one is the web distribution and the other one is rtmp rtmp stands for real time audio or video distribution it's basically used for distribution of a media content or a media file which are available in the s3 bucket here we are going to select a web distribution because primarily we will be using files which uses protocols http or the https so you have to click on get started and in the origin domain name you have to specify the bucket where your code is available so i have a bucket already created here you can see and what you need to do is you have to create a bucket with the url name or the domain name which you would be mapping with the route 53 service so this is a bucket that has already been created let me show you in a new tab so here you have to go to the storage under the storage you have to select the s3 bucket let's open the link in the new tab and let's look into how we can create the s3 buckets now here are a couple of buckets already created i have created one bucket with the domain name that i'm going to use and map it with the root 53 service so that is basically mapped to a region which is in ohio and if you open up this bucket here you will find an html web page the index.html has been already added right so similarly you have to create a bucket with a domain and an index.html page needs to be uploaded there now again we'll go to the cloud front we will try to create a distribution just have to click on create distribution select a web delivery method select an origin domain which is sunshinelearning.n and origin path you don't have to specify origin id is this one so basically when you define an origin domain name automatically the origin id appears you can customize this origin id as per your requirement also so rest of the things primarily we keep it as a default settings only until and unless if we require some customized settings to be done so let's say if you have to change the cache behavior settings you can do that otherwise we'll keep it as default now in the distribution setting you can either select use all edge locations for the best performance so what does aws basically do here it uses all the age locations which are associated with aws across the globe otherwise you can specify based on the regions also right apart from that if you want to enable firewalls or the access control list you can specify here and then what you need to do is in the default root object here you have to specify your index.html page which is in the s3 bucket the distribution state has to be enabled and if you want to use ipv6 as well you need to enable it click on create distribution now you can see here a distribution has been created it's in progress and it is enabled and it takes around 15 to 20 minutes to get that distribution completed the reason is that the web codes the web pages will be distributed across all the edge locations across the globe so hence it takes time to get that done right well let's move on to route 53 service and let's create the hosted zones so we'll type route 53 here scalable dns and domain name registration and what we are going to do here is we are going to map our url the domains pointed to the name servers that will be provided by the root 53 so we have to create a hosted zone let's wait for it so now the root 53 dashboard is open and you can see one hosted zone is already created so just click on the hosted zones and in order to point the traffic from the external domains towards the aws you have to first point the domains traffic to the hosted zone in the route 53 so i'll click on create hosted zone but before that i will first delete the existing one and then i'll create another record another hosted zone right put your domain name let's say i put as sunshine learning dot in and it is acting as a public hosted zone rest of the things will be default click on create hosted zone now it gives you four name servers and these four name servers has to be updated in the domain so you have to update these name servers in a platform from where you have purchased the domain right so this is half of the work done then what you need to do is you have to go and create records now in the records you can select a routing policy so right now what we are targeting we are targeting basically that the traffic from the domain should be pointed directly towards the cloudfront distribution hence we are going with a simple routing right now click on next here you have to specify the record sets so we are going to create the records uh just click on default simple record put here as world wide web and you have to select an end point so end point we are selecting for the cloudfront distribution so we have to specify alias for the cloudfront distribution now here we are going to put the cloudfront distribution url and then we are going to define the simple report set so what we need is we need a cloudfront distribution url which you can find it uh from the cloudfront service itself and you can find the domain name here itself you just have to copy that and then paste it here in the distribution and then just click on define simple records again click on create records and here you can see the record set has been updated now this domain is basically pointed towards these name servers which are further pointed towards the cloud front distribution right now the only thing which is left is that within the domain from wherever you have purchased the domain you should update these phone name servers and then you can see the live traffic coming on this domain will have an output from the cloud front distribution today's topic is on aws auto scaling so this is akil i would be taking up a tutorial on the auto scaling let's begin with our tutorial and let's look into what we have in today's session so i would be covering up why we require aws auto scaling what is aws auto scaling what are the benefits of using the scaling service how this auto scaling works the different scaling plans we have what is the difference between the snapshot and the ami what is the load balancer and how many types of load balancers we have and along with that i would be covering up a real life demo on the aws let's begin with why we require aws auto scaling now before aws cloud scaling there was a question in the mind of enterprises that they were spending a lot of money on the purchase of the infrastructure if they have to set up some kind of a solution so they have to purchase an infrastructure and one time cost was required so that was a burden for them in terms of procuring a server hardware software and then having a team of experts to manage all those infrastructure so they used to think that no longer they require these resources if there was a cost efficient solution for their project that was the project manager used to think now after the aws cloud scaling that was introduced automatically the auto scaling maintains the application performance based on the user requirements at the lowest possible price so what does the auto scaling does is that whenever there is a scalability required it manages it automatically and hence the cost optimization became possible now what is aws auto scaling let's look into deep so aws auto scaling is a service that helps users to monitor their applications and the servers and automatically adjust the capacity of their infrastructure to maintain the steadiness so they can increase the capacity they can even decrease the capacity also for the cost optimization and also predictable performance at the lowest possible cost now what are the benefits of auto scaling it gave the better fault tolerance applications you can get the servers created and you can have a clone copy of the servers so that you don't have to deploy the applications again and again better cost management because the scalability is decided by the aws automatically based on some threshold parameters it was a reliable service and whenever the scaling is created or initiated you can get the notifications onto your mail ids or to your cell phones scalability as i mentioned is always there in the auto scaling it can scale up it can scale down as well and it has the flexibility the flexibility in terms of whenever you want to schedule it if you want to stop it if you want to keep size of the servers at a fixed number uh you can always make the changes on the fly and the better availability now with the use of the auto scaling we come around with the terminology called snapshot and the ami let's look into the difference between the snapshots and the ami snapshots versus ami so in a company there was one of the employee that was facing an issue with launching the virtual machines so he asked his colleague a question is it possible to launch multiple virtual machines with a minimum amount of time because it takes a lot of time in terms of creating the virtual machines the other colleagues said that yes it is possible to launch multiple ec2 instance and that can be done at a lesser time and with the same configuration and this can be done either you use a snapshot or the ami on the aws then the colleague said that what are the differences between the snapshot and ami let's look into the difference now the snapshots basically kind of a backup of a single ebs volume which is just like a virtual hard drive that is attached to the ec2 instance whereas the ami it is basically used as a backup of an ec2 instance only the snapshots opts for this when the instance contain multiple static ebs when you opt for the snapshot whenever the instance contains multiple statics evs volumes ami this is widely used to replace the failed ec2 instance in the snapshots here you pay only for the storage of the modified data whereas with the ami you pay only for the storage that you use the snapshots are non-bootable images on ebs volume whereas ami are bootable images on the ec2 instance however creating an ami image will also create the evs snapshots now how does aws auto scaling work let's look into it so for the aws auto scaling to work you have to configure single unified scaling policy for application resource and this scaling policy with that you can explore the applications also and then select the service you want to scale also for the optimization select do you want to optimize cost or do you want to optimize the performance and then keep track of scaling by monitoring or getting the notifications now what are the different scaling plans we have so in the auto scaling a scaling plan basically helps a user to configure a set of instructions for scaling based on the particular software requirement the scaling strategy basically guides the service of aws auto scaling on how to optimize resources in a particular application so it's basically a kind of the parameters that you set it up so that how the resource optimization can be achieved in the auto scaling with the scaling strategies users can create their own strategy based on their required metrics and thresholds and this can be changed on the fly as well what are the two types of scaling policies we have so there are basically dynamic scaling and the predictive scaling now what is dynamic scaling it basically guides the service of aws auto scaling on how to optimize the resources and it is helpful in optimizing resources for availability and particular price now with scaling strategies users can create their plan based on the required metrics and thresholds so a metric can be like let's say a network in network out or it can be a cpu utilization memory utilization likewise now in the predictive scaling its objective is to predict future workload based on daily and weekly trends and regular forecast future network traffic so it is kind of a forecast that happens based on the previous past experiences it uses a machine learning technique for analyzing that network graphic and this scaling is like how weather forecast works right it provides schedule scaling actions to ensure the resource capacity is available for application requirement now with the auto scaling you would need the load balancers also because if there are multiple instances that are created then you would need a load balancer to distribute the load to those instances so let's understand what do we mean by a load balancer a load balancer basically acts as a reverse proxy and it is responsible for distributing the network or the application traffic across multiple servers with the help of a load balancer you can achieve a reliability you can achieve a fault tolerance of an application that is basically it increases the fault tolerance and the reliability so for example when there is a high network traffic that is coming to your application and if that much traffic comes to your application to the instances your instances may crash so how you can avoid that situation so you need to manage the network traffic that is coming to your instances and that can be done with the load balancer so thanks to the aws load balancers which helps in distributing network traffic across backend servers in a way that it increases performance of an application here in the image you can see the traffic coming from a different resources landing on to the ec2 instance and the load balancer is actually distributing that traffic to all the three instances hence managing the network traffic quite properly now what are the types of load balancers we have there are three types of load balancers on the aws one is the classic load balancer second is the application load balancer and the third one is the network load balancer let's look into what we have in the classic load manager so the classic load balancer is the most basic form of load balancing and we call it as a primitive load balancer also and it is widely used for the ec2 instances it is based on the ip address and the tcp port and it routes network traffic between end users as well as in between the backend servers and it does not support host-based routing and it results in low efficiency of resources let's look into what we have in the application load balancer this is one of the advanced forms of load balancing it performs a task on the application level in the osi model it is used when there are http and https traffic routing is required and also it supports the host based and pathways routing and performs well with the microservices or the backend applications the network load balancer performs the task at layer 4 of the connection level in the osi model the prime role of the network load balancer is to route the tcp traffic and it can manage a massive amount of traffic and is also suitable to manage the low latencies let's look into the demo and see how practically we can create the auto scale guys let's look into the demo for how we can create an auto scaling on the aws console so right now i'm logged into the aws console and i am in the mumbai region what you need to do is you have to go to the compute section and under that click on the ec2 service let's wait for the ec2 servers to come now just scroll down and under the load balancing there is an option called auto scaling so there first you have to create a launch configuration and then after that you have to create the auto scaling groups so click on launch configuration and then you have to click on create launch configurations so click on create launch configuration now this launch configuration is basically this set of parameters that you define while launching an auto scaling so that this uniformity is maintained with all the instances so that includes let's say if you select a windows os or linux os that particular type of an operating system will be implemented in all the instances that will be part of an auto scaling so there are certain set of parameters that we have to specify during the launch configuration so that we can have a uniformity in terms of launching the servers so here i would select an amazon linux ami and then i would select the type of server which will be t2.micro click on configure details put the name to the launch configuration let's say we put it as a demo and the rest of the things we'll keep it default click on add storage since it's a linux ami we can go with the 8gb storage that should be fine click on configure security group let's create a new security group which has the ssh port opened and that is open for anywhere which is basically source ipv4 and ipv6 ips any ip will be able to access that click on review just review your launch configuration if you want to make changes you can do that otherwise click on create a launch configuration you would need the key pair and this key pair will be a unique key pair which will be used with all the instances that are part of the auto scaling group so we can select an existing keypair if you have that otherwise you can create a new keypair so i have an existing keypair i'll go with that acknowledge it and click on create launch confusion now we have successfully launched the configuration of an auto scaling the next thing is to create an auto scaling group so click on create an auto scaling group using this launch configuration put a group name let's say we put something like test and the group size to start with uh it says one instance so that means at least a single instance will always be running and it will be initiated and running 24 cross 7 till the auto scaling is available you can increase the size of the minimum base instances also let's say you can change it to two also so you would get at least two servers running all the time so we'll go with the one instance the network would be the vpc default and uh in the vpc particular region we can select the availability zones so let's say if i select availability zone 1a and then availability is on one b so how the instances will be launched so one instance will be launched in one a the other one in the one b the third one in the one a fourth one and the one b likewise it will be equally spreaded among the availability zones next part is to configure the scaling policies so click on it if you want to keep this group at its initial size let's say if you want to go with only a single instance or two instances and you don't want the scaling to progress you can put it keep this group at its initial size so this is basically a way to hold the scaling but we'll use the scaling policies to adjust the capacity of this group so click on it and we would scale between let's say minimum one instance that we have and we'll scale it between one two four instances and what condition on what basis these instances will be scaled up or scaled down would be defined in the scale group size so the scaling policies you can implement based on a scale group size or using the simple scaling policies using the steps so in the scale group size you have a certain metrics you can use average cpu utilization you can define a metric related to average networking average network out or the load balancer request counts per target and if you create the simple scaling policies using steps then you need to create the alarms and there you can get some more metrics that you can add up as a parameter for the auto scaling let's go with the scaling group size let's go with the metric type as average cpu utilization and the target value here you have to specify what would be the threshold that when the instance cpu utilization is crossed then a new instance should be initiated so you can put a reasonable threshold for that let's say we put something like 85 percent and whenever the instance cpu utilization is crossed 85 threshold you will see that there will be a new instance created let's go to the next configure notifications and here you can add notifications so let's say if there is a new instance that is initiated and you want to basically be notified so you can get notifications over your email ids or you can get it on the cell phones so for that for adding the notification you would need the sns service that is called as a simple notification service and you have to create a topic there you have to subscribe for the topic using your email id and then you should get the notifications click on configure tags the tags are not mandatory you can basically put a tag let's say if you want to identify the instance for the purpose it was created otherwise you can leave it blank also click on review and review your scaling policies notification tags as well as the scaling group details click on create auto scaling group and here you go your scaling has been launched click on close and you should get at least a single instance initiated automatically by the auto scaling so let's wait for the details to appear so here you can see our launch configuration name demo auto scaling group name test minimum instance we want one the maximum instances we want four we have selected two availability zones ap south one ap south 1b and the instance one has been initiated and if you want to verify where exactly this instance has been initiated just click on the instances here and here you will see that our single instance has been initiated that is in service and that has been initiated in ap south 1b now once the threshold of this instance crosses 85 percent that is what we have defined in the scaling policies then you should see that another instance will be initiated so likewise this is basically i have created steps to initiate a scaling policy that means to increase the number of servers whenever the threshold crosses likewise here itself you can add another policy to scale down the resources in case if the cpu utilization goes to a normal value today we are going to discuss about amazon redshift which is one of the data warehouse service on the aws so let's begin with amazon redshift and let's see what we have for today's session so what's in it for you today we'll see what is aws uh why we require amazon redshift what do we mean by amazon redshift the advantages of amazon redshift the architecture of amazon redshift some of the additional concepts associated with the redshift and the companies that are using the amazon redshift and finally we'll cover up one demo which will show you the practical example that how you can actually use the redshift service now what is aws as we know that aws stands for amazon web service it's one of the largest cloud providers in the market and it's basically a secure cloud service platform provided from the amazon also on the aws you can create and deploy the applications using the aws service along with that you can access the services provided by the aws over the public network that is over the internet they are accessible plus you pay only whatever the service you use for now let's understand why we require amazon redshift so earlier before amazon readchef what used to happen that the people used to or the developers used to fetch the data from the data warehouse so data warehouse is basically a terminology which is basically represents the collection of the data so a repository where the data is stored is generally called as a data warehouse now fetching data from the data warehouse was a complicated task because might be a possibility that the developer is located at a different geography and the data data warehouse is at a different location and probably there is not that much network connectivity or some networking challenges internet connectivity challenges security challenges might be and a lot of maintenance was required uh to manage uh the data warehouses so what are the cons of the traditional data warehouse services it was time consuming to download or get the data from the data warehouse maintenance cost was high and there was the possibility of loss of information in between the downloading of the data and the data rigidity was an issue now how these problems could overcome and this was uh basically solved with the introduction of amazon redshift over the cloud platform now we say that amazon redshift has solved traditional data warehouse problems that the developers were facing but how what is amazon redshift actually is so what is amazon redshift it is one of the services over the aws amazon web services which is called as a data warehouse service so amazon redshift is a cloud-based service or a data warehouse service that is primarily used for collecting and storing the large chunk of data so it also helps you to get or extract the data analyze the data using some of the bi tools so business intelligence tools you can use and get the data from the redshift and process that and hence it simplifies the process of handling the large scale data sets so this is the symbol for the amazon redshift over the aws now let's discuss about one of the use case so dna is basically a telecommunication company and they were facing an issue with managing their website and also the amazon s3 data which led down to slow process of their applications now how could they overcome this problem let's say that so they overcome this issue by using the amazon redshift and all the company noticed that there was a 52 increase in the application performance now did you know that amazon redshift is basically cost less to operate than any other cloud data warehouse service available on the cloud computing platforms and also the performance of an amazon redshift is the fastest data warehouse we can say that that is available as of now so in both cases one is that it saves the cost as compared to the traditional data warehouses and also the performance of this redshift service or a data warehouse service the fastest available on the cloud platforms and more than 15 000 customers primarily presently they are using the amazon redshift service now let's understand some of the advantages of amazon redshift first of all as we saw that it is one of the fastest available data warehouse service so it has the high performance second is it is a low cost service so you can have a large scale of data warehouse or the databases combined in a data warehouse at a very low cost so whatever you use you pay for that only scalability now in case if you wanted to increase the nodes of the databases in your redshift you can actually increase that based on your requirement and that is on the fly so you don't have to wait for the procurement of any kind of hardware or the infrastructure it is whenever you require you can scale up or scale down the resources so this scalability is again one of the advantage of the amazon redshift availability since it's available across multiple availability zones so it makes this service as a highly available service security so whenever you create whenever you access redshift you create a clusters in the redshift and the clusters are created in the you can define a specific virtual private cloud for your cluster and you can create your own security groups and attach it to your cluster so you can design the security parameters based on your requirement and you can get your data warehouse or the data items in a secure place flexibility and you can remove the clusters you can create under clusters if you are deleting a cluster you can take a snapshot of it and you can move those snapshots to different regions so that much flexibility is available on the aws for the service and the other advantage is that it is basically a very simple way to do a database migration so if you're planning that you wanted to migrate your databases from the traditional data center over the cloud on the redshift it is basically uh very simple to do a database migration you can have some of the inbuilt tools available on the aws access you can connect them with your traditional data center and get that data migrated directly to the redshift now let's understand the architecture of the amazon redshift so architecture of an amazon redshift is basically it combines of a cluster and that we call it as a data warehouse cluster in this picture you can see that this is a data warehouse cluster and this is a representation of amazon redshift so it has some of the compute nodes which does the data processing and a leader node which gives the instructions to these compute nodes and also the leader node basically manages the client applications that require the data from the redshift so let's understand about the components of the redshift the client application of amazon redshift basically interact with the leader node using jdbc or the odbc now what is jdbc it's a java database connectivity and the odbc stands for open database connectivity the amazon redshift service using a jdbc connector can monitor the connections from the other client applications so the leader node can actually have a check on the client applications using the jdbc connections whereas the odbc allows a leader node to have a direct interaction or to have a live interaction with the amazon redshift so odbc allows a user to interact with live data of amazon redshift so it has a direct connectivity direct access of the applications as well as the leader node can get the information from the compute nodes now what are these compute nodes these are basically kind of a databases which does the processing so amazon redshift has a set of computing resources which we call it as a nodes and the nodes when they are combined together they are called it as a clusters now a cluster a set of computing resources which are called as nodes and this gathers into a group which we call it as a data warehouse cluster so you can have a compute node starting from 1 to n number of nodes and that's why we call that the redshift is a scalable service because we can scale up the compute nodes whenever we require now the data warehouse cluster or the each cluster has one or more databases in the form of a nodes now what is a leader node this node basically manages the interaction between the client application and the compute node so it acts as a bridge between the client application and the compute nodes also it analyzes and develop designs in order to carry out any kind of a database operations so leader node basically sends out the instructions to the compute nodes basically perform or execute that instruction and give that output to the leader node so that is what we are going to see in the next slide that the leader node runs the program and assign the code to individual compute nodes and the compute nodes execute the program and share the result back to the leader node for the final aggregation and then it is delivered to the client application for analytics or whatever the client application is created for so compute nodes are basically categorized into slices and each node slice is alerted with specific memory space or you can say a storage space where the data is processed these node slices works in parallel in order to finish their work and hence when we talk about a redshift as a fast faster processing capability as compared to other data warehouses or traditional data warehouses this is because that these node slices work in a parallel operation that makes it more faster now the additional concept associated with amazon redshift is there are two additional concepts associated with the redshift one is called as the column storage and the other one is called as the compression let's see what is the column storage as the name suggests column storage is basically kind of a data storage in the form of a column so that whenever we run a query it becomes easier to pull out the data from the columns so column storage is an essential factor in optimizing query performance and resulting in quicker output so one of the examples are mentioned here so below example show how database tables store record into disk block by row so here you can see that if we wanted to pull out some kind of an information based on the city address age we can basically create a filter and from there we can put out the details that we require and that is going to fetch out the details based on the column storage so that makes data more structured more streamlined and it becomes very easier to run a query and get that output now the compression is basically to save the column storage we can use a compression as an attribute so compression is a column level operation which decreases the storage requirement and hence it improves the query performance and this is one of the syntax for the column compression now the companies that are using amazon redshirt one is lya the other one is equinox the third one is the pfizer which is one of the famous pharmaceuticals company mcdonald's one of the burger chains across the globe and philips it's an electronic company so these are one of the biggest companies that are basically relying and they are putting their data on the red shift data warehouse service now in another video we'll see the demo for using the amazon redshift let's look into the amazon redshift demo so these are the steps that we need to follow for creating the amazon redshift cluster and in this demo what we will be doing is that we'll be creating an im role for the redshift so that the redshift can call the services and specifically we'll be using the s3 service so the role that we'll be creating will be giving the permission to redshift to have an access of an s3 in the read-only format so in the step one what we require we'll check the prerequisites and what you need to have is the aws credentials uh if you don't have that you need to create your own credentials and you can use your credit and the debit card and then in the step two we'll proceed with the im role for the amazon redshift once the role is created we will launch a sample amazon redshift cluster mentioned in the step 3 and then we will assign a vpc security groups to our cluster now you can create it in the default vpc also you can create a default security groups also otherwise you can customize the security groups based on your requirement now to connect to the sample cluster you need to run the queries and you can connect to your cluster and run queries on the aws management console query editor which you will find it in the redshift only or if you use the query editor you don't have to download and set up a sql client application separately and in the step six what you can do is you can copy the data from the s3 and upload that in the redshift because the redshift would have an access a read-only access for the s3 as that will be created in the im role so let's see how we can actually use the redshift on the aws so i am already logged into my account i am in north virginia region i'll search for red shift service and here i find amazon redshift so just click on it let's wait for the redshift to come now this is a redshift dashboard and from here itself you have to run the cluster so to launch a cluster you just have to click on this launch cluster and once the cluster is created and if you wanted to run queries you can open query editor or you can basically create queries and access the data from the red shift so that's what it was mentioned in the steps also that you don't require a separate sql client application to get the queries run on the data warehouse now before creating a cluster we need to create the role so what we'll do is we'll click on the services and we'll move to im role section so i am role i can find here under the security identity and compliance so just click on the identity access management and then click on create roles so let's wait for i am page to open so here in the im dashboard you just have to click on the rules i already have the role created so what i'll do is i'll delete this role and i'll create it separately so just click on create role and under the aws services you have to select for the redshift because now the redshift will be calling the other services and that's why we are creating the role now which other services that the redshift will be having an access of s3 why because we'll be putting up the data on the s3 and that is something which needs to be uploaded on the redshift so we'll just search for the redshift service and we can find it here so just click on it and then click on red shift customizable in the use case now click on next permissions and here in the permissions give the access to this role assign the permissions to this role in the form of an s3 read-only access so you can search here for the s3 also let's wait for the policies to come in here it is let's type s3 and here we can find amazon s3 read-only access so just click on it and assign the permissions to this role tags you can leave them blank click on next review put a name to your role let's put my redshift role and click on a create role now you can see that your role has been created now the next step is that we'll move to redchat service and we'll create one cluster so click on the services click on amazon redshift you can find that in the history section since we browsed it uh just now and from here we are going to create a sample cluster now to launch a cluster you just have to click on launch this cluster whatever the uncompressed data size you want in the form of a gigabyte terabyte or petabyte you can select that and let's say if you select in the form of gb how much db memory you want you can define it here itself this also gives you the information about the costing on demand is basically pay as you use so they are going to charge you 0.5 dollars per r for using the two node slices so let's click on launch this cluster and this will be a dc2 dot large kind of an instance that will be given to you it would be in the form of a solid state drive ssds which is one of the fastest way of storing the information and the nodes two are mentioned by default that means there will be two node slices and that will be created in a cluster you can increase them also let's say if i put three node slices so it is going to give us 3 into 0.16 db per node storage now here you have to define the master username password for your redshift cluster and you have to follow the password instructions so i would put a password to this cluster and if it accepts that means it does not give you any kind of a warning otherwise it is going to tell you about you have to use the ascii characters and all and here you have to assign this cluster the role that we created recently so in the available im rules you just have to click on my redshift role and then you have to launch the cluster if you wanted to change something in with respect to the default settings let's say if you wanted to change the vpc from default vpc to your custom vpc and you wanted to change the default security groups to your own security groups so you can switch to advanced settings and do that modification now let's launch the cluster and here you can see the redshift cluster is being created now if you wanted to run queries on this redshift cluster so you don't require a separate sql client you just have to follow the simple steps to run a query editor and the query editor you will find it on the dashboard so let's click on the cluster and here you would see that the red shift cluster would be created with the three nodes in the us east 1b availability zone so we have created the redshift cluster in the ohio region and now what we'll do is we'll see how we can create the tables inside the redshift and we'll see how we can use the copy command so that we can directly move the data uploaded on the s3 bucket to the redshift database tables and then we'll query the results of a table as well so how we can do that first of all after creating the redshift cluster we have to install sql workbench slash j this is not a mysql which is managed by oracle and you can find this on the google you can download it from there and then you have to connect this client with the redshift database how you can do click on file click on connect window and after connecting a window you have to paste the url which is a jdbc driver this driver link you can find it onto the aws console so if you open up a redshift cluster there you would find the jdbc driver link let's wait for it so this is our cluster created let's open it and here you can find this jdbc url and also make sure that in the security groups of a redshift you have the port 5439 open for the traffic incoming traffic you also need to have the amazon redshift driver and this is the link where you can download the driver and specify the path once you are done with that you provide the username and the password that you created while creating the redshift cluster click on ok so this connects with the database and now the database connection is almost completed now what we will be doing in the sql workbench we will be first creating the sales table and then in the sales table we'll be adding up the entries copied from the s3 bucket and then move it to the redshift database and after that we'll query the results in the sales table now whatever the values you are creating in the table the same values needs to be in the data file and i have taken up this sample data file from this link which is docs.aws.amazon.com redshift sample database creation and here you can find a download file ticketbb.zip file this folder has basically multiple data files sample data files which you can actually use it to practice uploading the data on the redshift cluster so i have extracted one of the files from this folder and then i have uploaded that file in the s3 bucket now we'll move into the s3 bucket let's look for the file that has been uploaded on the s3 bucket so this is the bucket sample and sales underscore tab dot text is the file that i have uploaded this has the entries data entries that will be uploaded using a copy command onto the redshift cluster now after executing after putting up the command for creating up the table then we'll use a copy command and copy command we have to define the table name the table name is sales and we have to define the path from where the data would be copied over to the sales table in the red shift now path is the s3 bucket and this is the redshift bucket sample and it has to look for the data inside the sales underscore tab.text file also we have to define the role arn that was created previously and once it is done then the third step is to query the results inside the sales table to check whether our data has been uploaded correctly on the table or not now what we'll do is we'll execute all these three syntax it gives us the error because we have to connect it again to the database let's wait for it execute it it's again gives us the error let's look into the name of the bucket it's redshiftbucketsample so we have two t's mentioned here right let's connect with the database again and now execute it so table sales created and we got the error the specified bucket does not exist uh redshift bucket sample let's view the bucket name redshift bucket sample let's copy that put it here connect to the window connect back to the database right and now execute it so table sales created uh the data in the table has been copied from the s3 bucket uh to sales underscore tab.text to the redshift and then the query of the results now the results from the table has been queried hi guys i'm raul from simply learn and today i'd like to welcome you all to the greatest debate of the century today i'm joined by two giants of the cloud computing industry they'll be going head to head with each other to decide who amongst them is better it's going to be one hello fight now let's meet our candidates on my left we have aws who's voiced by a picture hi guys and on my right we have microsoft azure who's voiced by anjali hey there so today we'll be deciding who's better on the basis of their origin and the features they provide their performance on the present day and comparing them on the basis of pricing market sharing options free tier and instance configuration now let's listen to their opening statements let's start with aws launched in 2006 aws is one of the most commonly used cloud computing platforms across the world companies like adobe netflix airbnb htc pinterest and spotify have put their faith in aws for their proper functioning it also dominates the cloud computing domain with almost 40 percent of the entire market share so far nobody has even gotten close to beating that number aws also provides a wide range of services that covers a great number of domains domains like compute networking storage migration and so much more now let's see what azure has to say about that azure was launched in 2010 and is trusted by almost 80 of all fortune 500 companies the best of the best companies in the world choose to work only with azure azure also provides its services to more regions than any other cloud service provider in the world azure covers 42 regions already and 12 more are being planned to be made azure also provides more than 100 services spanning a variety of domains now that the opening statements are done let's have a look at the current market status of each of our competitors this is the performance route here we have the stats for the market share of aws azure and other cloud service providers this is for the early 2018 period amazon web services takes up a whopping 40 of the market share closely followed by hdr at 30 and other cloud services adding 30 this 40 indicates most organizations clear interest in using aws we are number one because of our years of experience and trust we've created among our users sure you're the market leader but we are not very far behind let me remind you more than 80 percent of the fortune 500 companies trust azure with their cloud computing needs so it's only a match of time before azure takes the lead the rest of the 30 percent that is in aws or azure accounts to the other cloud service providers like google cloud platform backspace ibm softlayer and so on now for our next round the comparison round first we'll be comparing pricing we'll be looking at the cost of a very basic instance which is a virtual machine of two virtual cpus and 8gbs of ram for aws this will cost you approximately 0.0928 us dollars per hour and for the same instance in azure it'll cost you approximately 0.096 us dollars per hour next up let's compare market share and options as i mentioned before aws is the undisputed market leader when it comes to the cloud computing domain taking up 40 percent of the market share by 2020 aws is also expected to produce twice its current revenue which comes close to 44 billion dollars not to mention aws is constantly expanding its already strong roaster of more than 100 services to fulfill the shifting business requirements of organizations all that is great really good for you but the research company gardner has released a magic quadrant that you have to see you see the competition is now nectar between azure and aws it's only a matter of time before azure can increase from its 30 market share and surpass aws this becomes more likely considering how all companies are migrating from aws to azure to help satisfy their business needs azure is not far behind aws when this comes to services as well azure service offerings are constantly updated and improved on to help users satisfy their cloud computing requirements now let's compare aws and azure's free offerings aws provides a significant number of services for free helping users get hands-on experience with the platform products and services the free tier services fall under two categories services that will remain free forever and the others that are valid only for one year the always free category offers more than 20 services for example amazon sns sqs cloudwatch etc and the valid fourier category offers approximately 20 services for example amazon s3 ec2 elastic cache etc both types of services have limits on the usage for example storage number of requests compute time etc but users are only charged for using services that fall under the valid for a year category after a year of their usage a show provides a free tier as well it also provides services that belong to the categories of free for a year and always free there are about 25 plus always free services provided by azure these include app service functions container service active directory and lots more and as of the valid for a year there are eight services offered there's linux or windows virtual machines blob storage sql database and few more azure also provides the users with credits of 200 us dollars to access all their services for 30 days now this is a unique feature that azure provides where users can use their credits to utilize any service of a choice for the entire month now let's compare instance configuration the largest instance that aws offers is that of a whopping 256gb of ram and 16 virtual cpus the largest that azure offers isn't very far behind either 224 gbs of ram and 16 virtual cpus and now for the final round now each of our contestants will be shown facts and they have to give explanations for these facts we call it the rapid fire round first we have features in which aws is good and azure is better aws does not cut down on the features it offers its users however it requires slightly more management on the user's part azure goes slightly deeper with the services that fall under certain categories like platform as a service and infrastructure as a service next we have hybrid cloud where aws is good and azure is better okay although aws did not emphasize on hybrid cloud earlier they are focusing more on technology now azure has always emphasized on hybrid cloud and has features supporting it since the days of its inception for developers aws is better and azure is good of course it's better because aws supports integration with third-party applications well azure provides access to data centers that provide a scalable architecture for pricing both aws and azure are at the same level it's good for aws because it provides a competitive and constantly decreasing pricing model and in the case of azure it provides offers that are constantly experimented upon to provide its users with the best experience and that's it our contestants have finished giving their statements now let's see who won surprisingly nobody each cloud computing platform has its own pros and cons choosing the right one is based entirely on your organization's requirements hi guys today we've got something very special in store for you we're going to talk about the best cloud computing platform available amazon web services uh rahul i think you said something wrong here the best cloud computing platform is obviously google cloud platform no it isn't aws has more than 100 services that span a variety of domains all right but google cloud platform has cheaper instances what do you have to say about that well i guess there's only one place we can actually discuss this a boxing ring so guys i'm opexia and i will be google cloud platform and i'm rahul i'll be aws so welcome to fight night this is aws versus gcp the winner will be chosen on the basis of their origin and the features they provide their performance in the present day and comparing them on the basis of pricing market sharing options the things they give you for free and instance configuration now first let's talk about aws aws was launched in 2004 and is a cloud service platform that helps businesses grow and scale by providing them services in a number of different domains these domains include compute database storage migration networking and so on a very important aspect about aws is its years of experience now aws has been in the market a lot longer than any other cloud service platform which means they know how businesses work and how they can contribute to the business growing also aws has over 5.1 billion dollars of revenue in the last quarter this is a clear indication of how much faith and trust people have in aws they occupy more than 40 percent of the market which is a significant chunk of the cloud computing market they have at least 100 services that are available at the moment which means just about every issue that you have can be solved with an aws service now that was great but now can we talk about gcp i hope you know that gcp was launched very recently in 2011 and it is already helping businesses significantly with the suite of intelligent secure and flexible cloud computing services it lets you build deploy and scale applications websites services on the same infrastructure as google the intuitive user experience that gcp provides with dashboards wizards is way better in all the aspects tcp has just stepped in the market and it is already offering a modest number of services and the number is rapidly increasing and the cost for a cpu instance or regional storage that gcp provides is a whole lot cheaper and you also get a multi-regional cloud storage now what do you have to say on that i'm so glad you asked let's look at present day in fact let's look at the cloud market share of the fourth quarter of 2017. this will tell you once and for all that aws is the leader when it comes to cloud computing amazon web services contributes 47 of the market share others like rackspace or verizon cloud contribute 36 percent microsoft azure contributes 10 the google cloud platform contributes 4 and ibm software contributes 3 47 of the market share is contributed by aws you need me to convince you any wait wait wait all that is fine but we only started a few years back and have already grown so much in such a less span of time haven't you heard the latest news our revenue is already a billion dollars per quarter wait for a few more years and the world shall see and aws makes 5.3 billion dollars per quarter it's going to take a good long time before you can even get close to us yes yes we'll see now let's compare a few things for starters let's compare pricing for aws a compute instance of two cpus in 8gb ram costs approximately 68 us dollars now a computer instance is a virtual machine in which you can specify what operating system ram or storage you want to have for cloud storage it costs 2.3 cents per gb per month with aws you really want to do that because gcp wins this hands down let's take the same compute instance of two cpus with 8gb ram it will cost approximately 50 dollars per month with gcp and as per my calculations that's a 25 annual cost reduction when compared to aws talking about cloud storage costs it is only 2 cents per gb per month with gcp what else do you want me to say let's talk about market share and options now aws is the current market leader when it comes to cloud computing now as you remember we contribute at least 47 percent of the entire market share aws also has at least 100 services available at the moment which is a clear indication of how well aws understands businesses and helps them grow yeah that's true but you should also know that gcp is steadily growing we have over 60 services that are up and running as you can see here and a lot more to come it's only a matter of time when we will have as many services as you do many companies have already started adopting gcp as a cloud service provider now let's talk about things you get for free with aws you get access to almost all the services for an entire year with usage limits now these limits include an hourly or by the minute basis for example with amazon ec2 you get 750 hours per month you also have limits on the number of requests to services for example with aws lambda you have 1 million requests per month now after these limits across you get charged standard rates with gcp you get access to all cloud platform products like firebase the google maps api and so on you also get 300 in credit to spend over a 12 month period on all the cloud platform products and interestingly after the free trial ends you won't be charged unless you manually upgrade to a paid account now there is also the always free version for which you will need an upgraded billing account here you get to use a small instance for free and 5gb of cloud storage any usage above this always free usage limits will be automatically billed at standard rates now let's talk about how you can configure instances with aws the largest instance that's offered the 128 cpus and 4 tvs of ram now other than the on-demand method like i mentioned before you can also use spot instances now these are the situations where your application is more fault tolerant and can handle an interruption now you pay for the spot price which is effective at a particular r now these spot prices do fluctuate but are adjusted over a period of time the largest instance offered with google cloud is 160 cpus and 3.75 tbs ram like spot instances of aws google cloud offers short-lived compute instances suitable for bad jobs and fault tolerant workloads they are called as preemptable instances so these instances are available at 80 percent off on on demand price hence they reduce your compute engine costs significantly and unlike aws these come at a fixed price google cloud platform is a lot more flexible when it comes to instance configuration you simply choose your cpu and ram combination of course you can even create your own instance types this way before we wrap it up let's compare on some other things as well telemetry it's a process of automatically collecting periodic measurements from remote devices for example gps gcp is obviously better because they have superior telemetry tools which help in analyzing services and providing more opportunities for improvement when it comes to application support aws is obviously better since they have years of experience under their bed aws provides the best support that can be given to the customers containers are better with gcp a container is a virtual process running in user space as kubernetes was originally developed by google gcp has full native support for the tools other cloud services are just fine tuning a way to provide humanities as a service also the containers help with abstracting applications from their environment they originally run it the applications can be deployed easily regardless of their environment when it comes to geographies aws is better since they have a head start of a few years aws in this span of time has been able to cover a larger market share and geographical locations now it's time for the big decision so who's it going to be yeah who is it going to be gcp or aws i think i'm going to go for choosing the right cloud computing platform is the decision that's made on the basis of the user or the organization's requirement on that note i believe it's time for us to wrap this video up aws azure and gcp are three of the world's largest cloud service providers but how are they different from each other let's find out hey guys i'm rahul and i'll be representing amazon web services i'm chinmay and i'll be representing microsoft azure and i am shruti and i'll be representing google cloud platform so welcome to this video on aws versus azure vs gcp talking about market share amazon web services leads with around 32 of the worldwide public cloud share azure owns up to 16 of the worldwide market share and gcp owns around nine percent of the world's market share let's talk about each of these service providers in detail aws provides services that enable users to create and deploy applications over the cloud these services are accessible via the internet aws being the oldest of the lot was launched in the year 2006. azure launched in 2010 is a computing platform that offers a wide range of services to build manage and deploy applications on the network using tools and frameworks launched in the year 2008 gcp offers application development and integration services for its end users in addition to cloud management it also offers services for big data machine learning and iot now let's talk about availability zones these are isolated locations within data center regions from which public cloud services originate and operate talking about aws they have 69 availability zones within 22 geographical regions this includes regions in the united states south america europe and asia pacific they are also predicted to have 12 more editions in the future azure available in 140 countries has over 54 regions worldwide grouped into six geographies these geographical locations have more than 100 data centers gcp is available in 200 plus countries across the world as of today gcp is present in 61 zones and 20 regions with osaka and zurich being the newly added regions now let's talk about pricing these services follow the pay-as-you-go approach you pay only for the individual services you need for as long as you use them without requiring long-term contracts or complex licensing now on screen you can see the pricing for each of these cloud service providers with respect to various instances like general purpose compute optimized memory optimized and gpu now let's talk about the compute services offered first off we have virtual servers for aws we have ec2 it is a web service which eliminates the need to invest in hardware so that you can develop and deploy applications in a faster manner it provides virtual machines in which you can run your applications azure's virtual machines is one of the several types of computing resources that azure offers azure gives the user the flexibility to deploy and manage a virtual environment inside a virtual network gcp's vm service enables users to build deploy and manage virtual machines to run workloads on the cloud now let's talk about the pricing of each of these services aw cc2 is free to try it is packaged as part of aws's free tier that lasts for 12 months and provides 750 hours per month of both linux and windows virtual machines azure virtual machine service is a part of the free tier that offers this service for about 750 hours per month for a year the user gets access to windows and linux virtual machines gcp's vm service is a part of a free tier that includes micro instance per month for up to 12 months now let's talk about platform as a service or pass services for aws elastic bean stock is an easy to use service for deploying and scaling web applications and services developed with java.net node.js python and much more it is used for maintaining capacity provisioning load balancing auto scaling and application health monitoring the pass backbone utilizes virtualization techniques where the virtual machine is independent of the actual hardware that hosts it hence the user can write application code without worrying about the underlying hardware google app engine is a cloud computing platform as a service which is used by developers for hosting and building apps in google data centers the app engine requires the apps to be written in java or python and store data in google bigtable and use the google query language for this next let's talk about virtual private server services aws provides light sale it provides everything you need to build an application or a website along with the cost-effective monthly plan and minimum number of configurations in simple words vm image is a more comprehensive image for microsoft azure virtual machines it helps the user create many identical virtual machines in a matter of minutes unfortunately gcp does not offer any similar service next up we have serverless computing services aws has lambda it is a serverless compute service that lets you run your code without facilitating and managing servers you only pay for the compute time you use it is used to execute backend code and scales automatically when required azure functions is a serverless compute service that lets you run even triggered code without having to explicitly provision or manage infrastructure this allows the users to build applications using serverless simple functions with the programming language of their choice gcp cloud functions make it easy for developers to run and scale code in the cloud and build image driven serverless applications it is highly available and fault tolerant now let's talk about storage services offered by each of these service providers first off we have object storage aws provides s3 it is an object storage that provides industry standard scalability data availability and performance it is extremely durable and can be used for storing as well as recovering information or data from anywhere over the internet blob storage is an azure feature that lets developers store unstructured data in microsoft's cloud platform along with storage it also offers scalability it stores the data in the form of tears depending on how often data is being accessed google cloud storage is an online storage web service for storing and accessing data on google cloud platform infrastructure unlike the google drive google cloud storage is more suitable for enterprises it also stores objects that are organized into buckets amazon provides ebs or elastic block store it provides high performance block storage and is used along with ec2 instances for workloads that are transaction or throughput intensive azure managed disk is a virtual hard disk you can think of it like a physical disk in an on-premises server but virtualized these managed disks allow the users to create up to 10 000 vm disks in a single subscription persistent storage is a data storage device that retains data after power to the device is shut off google persistent disk is durable and high performance block storage for gcp persistent disk provides storage which can be attached to instances running in either google compute engine or kubernetes engine next up we have disaster recovery services aws provides a cloud-based recovery service that ensures that your it infrastructure and data are recovered while minimizing the amount of downtime that could be experienced site recovery helps ensure business continuity by keeping business apps and workloads running during outages it allows recovery by orchestrating and automating the replication process of azure virtual machines between regions unfortunately gcp has no disaster recovery service next let's talk about database services first off for aws we have rds or relational database service it is a web service that's cost effective and automates administration tasks basically it simplifies the setup operation and scaling of a relational database microsoft azure sql database is a software as a service platform that includes built-in intelligence that learns app patterns and adapts to maximize performance reliability and data protection it also eases the migration of sql server databases without changing the user's applications cloud sql is a fully managed database service which is easy to set up maintain and administer relational postgresql mysql and sql server databases in the cloud hosted on gcp cloud sql provides a database infrastructure for applications running anywhere next we have nosql database services aws provides dynamodb which is a managed durable database that provides security backup and restore and in-memory caching for applications it is well known for its low latency and scalable performance azure cosmos db is microsoft's globally distributed multi-model database service it natively supports nosql it natively supports nosql created for low latency and scalable applications gcp cloud data store is a nosql database service offered by google on the gcp it handles replication and scales automatically to your application's load with cloud data stores interface data can easily be accessed by any deployment target now let's talk about the key cloud tools for each of these service providers for aws in networking and content delivery we have aws route 53 and aws cloud front for management we have aws cloud watch and aws cloud formation for development we have aws code start and aws code build for security we have iam and key management service for microsoft azure networking and content delivery we have content delivery network and express route for management tools we have azure advisor and network watcher for development tools for management we have azure advisor and network watcher for development we have visual studio ide and azure blob studio for security we have azure security center and azure active directory for gcp we have the following tools for networking and content delivery we have cloud cdn and cloud dns for management we have stack driver and gcp monitoring for development we have cloud build and cloud sdk and finally for security we have google cloud im and google and cloud security scanner now let's talk about the companies using these cloud providers for aws we have netflix unilever kellogg's nasa nokia and adobe pixar samsung ebay fujitsu emc and bmw among others use microsoft azure so as seen on your screens the companies that use gcp are spotify hsbc snapchat twitter paypal and 20th century fox let's talk about the advantages of each of these services amazon provides enterprise friendly services you can leverage amazon's 15 years of experience delivering large-scale global infrastructure and it still continues to hone and innovate its infrastructure management skills and capabilities secondly it provides instant access to resources aws is designed to allow application providers isvs and vendors to quickly and securely host your applications whether an existing application or a new sas based application speed and agility aws provides you access to its services within minutes all you need to select is what you require and you can proceed you can access each of these applications anytime you need them and finally it's secure and reliable amazon enables you to innovate and scale your application in a secure environment it secures and hardens your infrastructure more importantly it provides security at a cheaper cost than on-premise environments now talking about some of the advantages of azure microsoft azure offers better development operations it also provides strong security profile azure has a strong focus on security following the standard security model of detect assess diagnose stabilize and close azure also provides a cost-effective solution the cloud environment allows businesses to launch both customer applications and internal apps in the cloud which saves an it infrastructure costs hence it's opex friendly let's now look at the advantages of gcp google builds in minute level increments so you only pay for the compute time you use they also provide discounted prices for long running workloads for example you use the vm for a month and get a discount gcp also provides live migration of virtual machines live migration is the process of moving a running vm from one physical server to another without disrupting its availability to the users this is a very important differentiator for google cloud compared to other cloud providers gcp provides automatic scalability this allows a size container scale to as many cpus as needed google cloud storage is designed for 99.9 percent durability it creates server backup and stores them in an user configured location let's talk about the disadvantages of each of these services for aws there's a limitation of the ec2 service aws provides limitations on resources that vary from region to region there may be a limit to the number of instances that can be created however you can request for these limits to be increased secondly they have a technical support fee aws charges you for immediate support and you can opt for any of these packages developer which costs twenty nine dollars per month business which costs more than hundred dollars an enterprise that costs more than fifteen thousand dollars it has certain network connectivity issues it also has general issues when you move to the cloud like downtime limited control backup protection and so on however most of these are temporary issues and can be handled over time talking about some of the disadvantages of microsoft azure code base is different when working offline and it requires modification when working on the cloud pass echo system is not as efficient as iaas azure management console is frustrating to work with it is slow to respond and update and requires far too many clicks to achieve simple tasks azure backup is intended for backing up and restoring data located on your on-premises servers to the cloud that's a great feature but it's not really useful for doing bare metal restores of servers in a remote data center let's now look into the disadvantages of gcp so when it comes to cloud providers the support fee is very minimal but in the case of gcp it is quite costly it is around 150 dollars per month for the most basic service similar to aws s3 gcp has a complex pricing schema also it is not very budget friendly when it comes to downloading data from google cloud storage hey guys this is chidanan from simply learn and i welcome you to this short tutorial and demo on kubernetes kubernetes specific to aws cloud platform so what's a part of this tutorial and demo what's in store for you at offset i would like to cover the basics of orchestration tools as most of you would know kubernetes is one of the most popular orchestration tools in recent times specifically for applications which are cloud native and deployed on some or the other types of containers but what are these organization tools why would one need to use orchestration tools what are the facilities or features provided by this orchestration tools that's the first thing that i'm going to cover after that i will pick two orchestration tools specific to container management docker swarm versus kubernetes i'm going to compare them with regard to the features that they provide the use cases that they cover what facilities that they provide and when to use what after that i will get into a little bit details of the kubernetes architecture what exactly is required for setting up a kubernetes cluster what runs in a control plane or a master node what runs as a part of the worker node after that i will end the tutorial by running you through a demo by setting up a three node kubernetes cluster on aws platform i will use something called as cops which is one of the admin tools for setting up production grade kubernetes cluster so i will use this to set up a three note cluster on aws all right now that we set the context right let me get started so what are orchestration tools the application development and the lifecycle management from the time the application is developed connecting the source code to your continuous integration to testing them all along your development process and eventually moving it down to production managing your production servers the dependencies that your software has the requirement that it has in terms of the hardware the features of fault tolerance self-healing capabilities auto scaling all this has complicated over the last few years as a part of devops one thing that everyone is interested in is managing all this application dependency or life cycle management using some or the other kind of a tool so that's where these orchestration tools have become very very popular so what kind of features that they provide is you'll have to just let them know what are the kinds of tool sets that is required what are the fault tolerance mechanisms that it has to be adopted to what kind of a self-healing capabilities that application will have to need and if at all there is any auto scaling features that is required if at all you can bake in all these specific parameters into your tool your orchestration tool becomes a one-stop shop for all your deployment and configuration management needs that's where this orchestration tools have become a very very popular and relevant specifically these days when people are more and more interested in adopting devops practices all right so having said that about orchestration tools let's concentrate on two of these orchestration tools specifically for container management docker swarm and kubernetes many of the applications that are written for these kind of containers are called or kind of fall into a category of cloud native where the application need not know much about the underlying infrastructure or the platform where these applications are running so these two along with apache missions there are many other container management systems but i'm going to pick docker swarm and kubernetes for comparing the features that is provided by these two frameworks for the sake of comparison i picked up docker swarm and kubuntes just because of the reason that both of them operate in the space of container management they do something very very similar to containers that's the similarity that exists between these two orchestration tools however there's a big difference that exists when it comes to the types of containers that both of them cater to and also the capacity of workloads they can be used for docker swamp is a cluster management solution that is provided by docker container docker is one of the very very popular containers of recent times and in case you have your applications that is totally powered by only docker containers if you have a need where you want to run a cluster of servers which are running only docker containers docker swarm should be your choice for your orchestration tool kubernetes on the other hand can cater to any other types of containers other than docker and including docker as well so in case you have your applications which have got docker in them which have got rkt in them which they have got alex in them or any other type of container kubernetes should be your choice of orchestration too a docker swarm can manage up to 40 50 or 60 max nodes so in case your application is totally written in docker containers and the load or the expected cluster size is about 50 60 nodes docker swarm should be your choice of orchestration tool on the other hand kubernetes is something that was open sourced by google and the kind of the scale of operations that kubernetes can cater to is google scale so if in case your applications are more than a thousand nodes that is where kubernetes comes into play that's the big difference that exists between docker swarm and kubernetes putting those differences aside let me compare docker swarm and kubernetes based upon other features which are similar in nature the first and important feature that kubernetes provides is something called as auto scaling if at all the load on your cluster is too high and your application is experiencing more load so kubernetes can add new nodes to your cluster of course you'll have to configure kubernetes in order to have those capabilities where it can spin up new vms or new nodes if at all you do that configuration correctly kubernetes has got the capacity or it has got the feature where it can bring up a new node and add it to the cluster on the fly based upon the load that exists at that moment on similar lines if at all the load is not too high it can identify a few of those nodes which have got less number of replicas or less number of application containers running on it it can move them to some other node and also scale down by deleting few nodes from your cluster so that is the powerfulness of auto scaling which is provided by kubernetes and this unfortunately does not exist in docker swamp the other feature of load balancing specifically application node balancing so docker swarm gives you an application level auto load balancing however kubernetes gives you the flexibility of manually configuring any other type of load balancer of your choice for application load balancing installation as i mentioned earlier docker swarm is a loose cluster of containers which are running on nodes it is very easy to spin up a new node and then connect them to a swarm and this can be done in a very very loosely coupled way where you can create swarms of your choice nodes are allowed to connect to swarms and quit or leave the swamps on the fly so in and all the installation is pretty easy and fast kubernetes on the other hand the configuration of kubernetes the way to spin up a big cluster specifying the size of the node how many masternodes how many config planes that you want how many nodes that you want it's a pretty tough thing to bring up a kubernetes scalability kubernetes strength is very very strong they are very tightly coupled and even they have the capability where on the cluster size things or the nodes can be increased on the fly based upon the requirement on a darkest form the cluster strength is weak as compared to copper days worker notes can be added to a swarm worker notes can be asked to leave us form or can be taken out of a swarm based upon the requirement so this is kind of a loosely coupled architecture for docker swarm while coverage is the cluster is very tightly coupled since docker swarm runs only docker containers containers find it very easy to share data a lot of other things with each other because they all have the same signature they are all from the same family so it's very easy for them to share not just volumes but a lot of other things however for kubernetes since it manages containers of different types if your application has to share some data across different containers there's a little bit of a complexity in how you would want your containers to share data also when it comes to kubernetes kubernetes groups containers in something called as pods while a pod can have either one container that's the preferred choice or multiple containers and the idea of pod is like each part can run in any other node it's not guaranteed that you would want to have two or three parts to be running on the same note that makes data sharing a little bit of a different thing compared to docker's form uh gui so there's not a good enough ui tool in docker's form at least the ce edition of it which will allow you to get to know what is running on your containers what is the size of the containers what is the volume of the containers and all that stuff there are some free and open source tools like proteiner which gives you a good visibility into your running docker containers however there's nothing at a swarm level that is provided by docker kubernetes gives you an out of the box dashboard which is easy to configure and set up you can also club it with some metrics services and you can also get information about the size of the cluster that is running what is the load on the nodes and stuff like that all this across your cluster in a very beautiful dashboard perspective so that way um this is little bit of a good ui for kubernetes while for docker's form there isn't anything that is provided out of the box let me spend some time in explaining a little bit about the kubernetes architecture what comprises of the kubernetes cluster what resides in a master or a control plane and what resides in a worker node this is a very high level uh depiction of a kubernetes cluster so this is the master node which has got something called as a cluster store a controller a scheduler component and an api server and these are the bunch of nodes that are connected or administered by the master node the master node can be one three five typically an odd number this is as per the typical cluster management thing where you would need the masternodes to be in odd numbers so that whenever there's any contention in terms of what needs to be deployed where or where to give the job to whom and stuff like that all the masters would cast their vote and they would decide on the outcome so that's the master node and there are a bunch of nodes which can be connected and administered by the master node cuba ctl or the command from using which anybody can assign some workloads to the cluster can also be run either on the same node or on a separate node so cubectl is the command line tool that we will be installing and using in order to fire up our commands to our cluster so if at all i have to put up a job on our cluster if i have to give out any specific commands to my cluster all this is done using cuba ctl there's a bunch of rest apis which is used by cuba ctl and cubectl will talk to the api server and fire up the commands specific to my cluster what runs in the master node master node without saying it's the most important component of any of the cluster if at all you're running um cluster with only one master node if your master node goes down there's no other way that you know you can or any user can talk to the different nodes in the cluster so master node is the most vital component responsible for the complete kubernetes cluster there's always one node that should be running as a master node that's the bare minimum requirement for your cluster so what are the other components in the master node the most important one is called etcd so this is nothing but a data store a value of key value pair which is stored which contains all information about your cluster so all the configuration details which node is up which worker node is down all this information is stored in the cluster store so all the managers would access this cluster store before they go ahead and decide any other work item or anything else that has to be done specific to your cluster what's in the controller as the name says controller is something it's like a daemon server that is running in a continuous loop all the time it is responsible for controlling the set of replicas the kind of workloads that is running based upon the configuration that the user has set in so if at all if you are aware of something called as replication controllers endpoint controllers namespace controllers all these controllers are managed by the controller component if any user asks for some three replicas of a particular pod or a service to be running and if at all any of the nodes goes down or the part goes on for whatsoever reason the controller is the one who wakes up and assigns this particular job or the part to some other available node by looking up for the details using in the cluster store that's the importance of the control manager or the controller the scheduler the scheduler assigns the tasks based upon whoever is asked for any job to be scheduled based upon a time frame or based upon some criteria it also tracks the working load as to what what exactly is the load who is running what and in the cluster and places the workload on whoever is the available resource at that time all right api server this is one other important component in in our kubernetes cluster where how would the end user deploy or give out any sort of a workload onto your cluster all the requests come to the api server so this is an entry point for all your requests that come into your cluster anybody wants to deploy something anybody want to scale up a controller anybody who wants to bring down a few services anybody wants to put in a service all this will have to come in as a part of a rest um api endpoint and api server is the entry point in case you don't want uh you know somebody to access your cluster in case you want only specific people or specific users to be running some specific workloads you can set all those role-based access control for this apa server so this is the entry point for anyone who wants to submit any job to your cluster so that's a quick overview about the master node what what are the important components of a master node now let's go over to what runs in a worker or a slave node as i mentioned earlier slave nodes are something where the job that is submitted to your cluster is eventually run as a pod as a service as a container so in kubernetes world there is something called as a pod a pod is nothing but a combination of a container it is a wrapper around your running containers so slave nodes or the worker nodes typically run these parts so that is where the whole workload typically gets run but there are a bunch of other components in the in the node which also manages what runs in the pod who has to have access to the pod what is the state of the pod is it in a running state is it going down for some reason and all the stuff so let's quickly go over those components that is there in the slave node the most important component in my opinion that should be on the on the node should be the container runtime as i mentioned earlier kubernetes can run any different types of containers not just docker so in case you want to have a node which wants to have docker running on it rkt running or lkc running on it or any other container environment that is running on it you have to ensure that the specific container runtime environment is available on that specific node so that whenever a job is submitted description of what needs to be run as a part of the pod what should be the image with it should be powered and what kind of a container to spin up so that's what you know when the job is finally assigned to this particular node it would definitely need a container runtime to be up and running so that exists only if at all the container runtime is installed and running on your kubernetes worker node okay now that we know what our kubernetes node can run how would somebody assign a job to our node that is using something called as a cubelet as the name says cubelet or cubelet is a small subset which talks to the cube api server so any node which has to run any kind of a pod all these instructions are passed around by the qbps server to the cube light and cubelet is capable of processing whatever job that is assigned to it and ensuring that so many parts and so many services are spun up based upon the requirement the last component that exists in the worker node is the cube proxy or kubernetes proxy this plays a very very important role acting as a load balancer and a network proxy so what typically happens is whenever the pods are running in nodes the parts can be typically running in any node there is no um affinity towards any node on which these parts are running because pods or containers are something called as ephemeral they can run anywhere so how would somebody reach out to these applications that is running in some pod which is running in one container now and running in probably another container another node altogether tomorrow so that is where your kubernetes proxy contains a picture and this component will ensure that any container that is spun up or any part that is spun up it keeps track of all these parts and kind of connects the endpoints or acts like a dns server so that it knows when somebody is trying to reach out to this particular service which is the part on which the service is typically running so that plays a very very important role by acting as a load balancer and a network proxy now that at a very high level we know what are all the components that make up of our kubernetes cluster let's go ahead and create a small cluster and spin up this cluster on aws platform finally we get to the demo section of my tutorial now that you guys are aware as to what is kubernetes what is the use of kubernetes you also know at a very very high level what are the components of a cluster what is a master node or a control plane what are the components of a master node and what should be existing in a worker node you're probably thinking that it's maybe pretty hard to kind of set this up so let me demystify that by running you with a quick demo of how to set up a simple node kubernetes cluster using a tool called cops and i will be doing this whole setup on my aws cloud okay so what is cops cops is nothing but a simple tool this is an admin tool that allows you to bring up production grade kubernetes environments as i said setting up a kubernetes cluster on a bare metal is little challenging so that is why i would use something called as a cops and i will use this on my cluster which i'll spin up on my aws instance so i have an aws account and what i will do first is i will first spin up an ec2 server i will power this with one of the um linux amis and i will install cops on it now this will be my starting point from where i'm going to trigger running up a big cluster in our case i'll try to keep it little small i don't want too many nodes in my cluster i'm going to have one master node and two worker nodes this whole operation would be done programmatically by installing this cops admin tool so as you may know aws is little hard for me to programmatically run other components or rather bring up servers it's not pretty simple so what i would do is i will create an iam role which will attach to my ec2 server so that my ec2 server gets powered with all the required role so that it can go ahead and spin up cluster based upon my configuration that i am going to specify all right so once i have my cluster set up i will also install cuba ctl which uh you would probably know by now is nothing but a command line utility using which i can kind of connect to my cluster give it some jobs put some parts and stuff like that so i will use cuba ctl i will also have a pair of ssh keys so that from this machine i would be able to successfully get onto the master node and submit jobs on my behalf so let me begin by first logging into my aws account all the steps that i will be following to bring up my aws cluster kubernetes cluster will be documented and i'll be sharing you this document in a github repository so that in case anybody wants to try you'll be more than happy to find all the required instructions in one place okay now the first thing that i would need is to launch an aws ec2 instance now this is an instance where i'm going to install cops and i will use it as a base server from where i am going to find some commands which will bring up my kubernetes cluster so let me log into my aws account and let me stick to one particular region and let me bring up some a one ec2 instance all right so let me launch my instance let me choose um doesn't matter in the ways but in case you want to try it on your free tire i would recommend choose free tire only and choose an ami of your choice let me choose this instance i will make it t2 micro that is good for me configuration instance details all right i don't want to change anything here that looks good maybe i will change this 30 gig that's good for me let me add a tag to my instance i will name this as my cops server all right i would need a bunch of posts to be opened up so what i'll do is i'll create a new security group called open to the world i would not recommend that you do this since i don't want to it will take a long time for me to specifically to pick and choose the ports that has to be open for running my cluster i don't want to do that i will open up all the http and https ports so that it's quicker so i will say all tcp i would say from anywhere just to be on the safer side also open http and https and i will make this anywhere and anywhere this is good for me i will say review and launch all right so i chose a t2 micro instance just for the sake of easiness i open up all the ports and um the instant details are you've seen what is that i've chosen all right now important part is like i would need a keyboard i need to create a new keypad i will call this as simply learn keys i will download this keypair and then i will launch the instance it will take some time for my ec2 instance to come up in the meanwhile what i'll do is i will convert i have this pen file which is the pair of keys the ssh keys with which i need to login to my ec2 instance so i'll need to convert this because i'm trying to connect from my windows box i would need to convert this into a ppk file so i have puttygen i'm going to load my set of keys here and convert that key into a ppk key all right open all right i would say save hang on save private key yes i would say simply learn or simply learn private private key and save this here and that that's all so this is my public key which is the pem file and this is my tpk or my private key now using the private key i will log into my ec2 instance okay now my ec2 instance is up and running it is running in this ohio region and this is the ip address of my public eye partners of my machine so let me connect to it so i use a moba x term which is nothing but an ssh emulator you can use putty or any of these sessions which allow you to do an ssh this is my ip address of my machine and since i've spun up an amazon ami the user this is the default user i would need to specify the private keys for my session and this is the private key for my session say okay i am in my amazon ec2 instance so let me look at what are the other things that i need to do all right so as i said i just brought up my ec2 instance but i would need my ec2 instance to run few things on my behalf so that it can spin up easy to other ec2 instances it can also talk to an s3 bucket where i'm going to store the instant state of my kubernetes cluster also some sort of an auto scaling group because i want to spin up more and more instances i also want to have a private hosted zone using my route 53 so i would need my ec2 instance to have all these sort of permissions for my server so what i would do is i will go and create a permission a rule in the ec2 rather in the aws im role and i will attach this i am role to my ec2 instance all right so let me go to my iam which is nothing but the identity and access management i will create a role called as possibly corpse role i will create a role this role will be used by my ec2 so i'm going to click on ec2 and say next permissions they would need a lot of permissions uh specific permissions to be very honest and the permissions are all listed out here s3 ec2 and all this stuff just to keep it pretty simple what i would do is i'll create a role with the administrative access all right so i don't want any tags i will review i'll create a row name i will say cops fold for ec2 alright so i'm going to create a rule which has got administrative access rules i'm going to create a rule so this would ensure that my ec2 instance from which i'm going to run my cluster would have all the prerequisite permissions so that it can go and spin up instances talk to us three buckets and all that stuff all right now let us have our running instance get powered by this role so i will connect this role to my running instance all right so this is my um easy to instant that is running i'll say action attach attach attach attach replace im rules so there is no rule as of now i would want my cops roll for easy this row that i created so i want this new room to be assigned to my ec2 all right great now my ec2 instance which is my cop server has got all the required permissions so that he can create a cluster on my behalf great now before this let me do a sudo yum update iphone y so that any of the because this is a newly provisioned vm i just want to ensure that all my library system libraries are all updated so that when i do install cop and cube ctl and all those things none of them will fail because of some dependency or packet issues so i'm just running a sudo update iphone y so that all the libraries are updated okay looks like it is done so let me go ahead and install cops cups is a pretty simple installation it is available in a particular location i have to just copy this and ensure that i do a curl and i install this particular stuff all right so it is fetching uh the cops tool is installing for me once it is copied down let me change the mode so that i can execute it and then also let me move it to my user local bin so that it is available for me in my path okay that's pretty much the corpse installation let me also do one other thing let me install something called as cuba ctl so this is what would be you know a tool using which i'm going to be firing my uh commands to my kubernetes cluster once it comes up right this is a pretty smaller executable so ctl all right so i have cuba ctl as well as cops installed on my ec2 server now okay now what i'll do next is i'm going to create an s3 bucket in aws so that the equivalent is a cluster state is persisted in this bucket so let me create a simple bucket with this name s3 bucket with this name so let me go to s3 and let me create a simple bucket with this name i will just say create a bucket okay so the simply learn.kubernetes is the bucket that i created so this bucket will store all information about my cluster let me now uh go and create a dns entry or a dns zone or aws route 53 hosted zone for my cluster so this is required so that you know i can give a cluster name for my kubernetes cluster that i'm coming up so i will create a private hosted zone and possibility possibly i will call this as simply learn dot in so this will be my name of my private hosted zone in case you already have any other public hosted zones in your name you can always put a route 53 hosted zone specific for your domain name since i don't have anyone i'm just going to create a simple private hosted zone so let me head down to route 53 click on any of these and you'll get these hosted zones out here so i'm going to create a hosted zone here i'm going to create a hosted zone click on create hosted zone um this would be simply learn lot in i want to create a public no i don't want public i want a private hosted zone and i'm going to associate my vpc id for this hosted zone since i'm going to try out all my exercises in the ohio region i will associate the vpc of the ohio region for this all right so this is the one that is specific to the ohio region so i will say this one i'll say create a hosted zone great now let me come back to my ec2 box my ec2 is all powered with uh whatever it needs in order to uh go ahead with the installation of my cluster only a few things that i need to take care of is that you know now that i put a name for my cluster i also have an s3 bucket that i've configured for my cluster i will have to ensure that i need to put in these two configurations as a part of my cluster building activity so i will open my bash rc file and i'm going to export these two variables if at all you remember well these two variables are nothing but the cluster name which is nothing but the public oh sorry the private hosted zone that i created and the s3 bucket where i'm going to create or rather i'm going to store my cluster state so let me copy these two and open up my bashrc file all right so i will just add these two and copy these add export them out as my variable i'm going to save this here now let me ensure that this gets picked up all right i've got these two that have configured and i also ensure that these environment variables are set now let me create a pair of ssh keys this is to ensure that i can log into the box that i'm going to be provisioning as a part of my cluster ssh hyphen keygen i don't want to give any parameters let it go to the home directory with the default name and without any passphrase all right so i created a bunch of my keypair now i'm all set to go ahead and run my cops now cops will ensure that this will take some time for the cluster to come up but this is exactly how i would define my cops command how do i have copied here so corpse create cluster the state of the cluster will be stored in this particular variable which is what i have exported out the number of node count is two node count is the worker node count if at all i don't specify configuration for specifying the master or the control pane as well if i don't specify that the default one is one so i'm going to create one primary or the master node and two uh worker nodes uh size of my master node size of the worker nodes uh what are the zones where i want this to be created and where do i store the cluster information and uh okay i've already added this master account this is actually not required but this is the command that i'm going to fire off so that i bring up my cluster all right that went off uh very fast but um this actually did not create a cluster it is just the definition of a cluster and that's why this came out very fast now that everything looks good with uh whatever configuration that i specified now let me go ahead and create the cluster by saying corpse update cluster iphone iphone 8 yes now this will take some time a good five to ten minutes for it to because this will actually start provisioning the servers and as i mentioned earlier based upon my configuration i'm going to come up with one master server and two nodes or the worker nodes all right so this is the command for me to validate the cluster and i'm going to try it out first i'm pretty sure that it will fail because all my servers are not up yet it is taking some time the validation everything failed but let me try to look at my ec2 instances and see how many servers do i have running as of now if you see i had only one server that i had started which was my cop server and automatically the other three instances one is called not start simply not in this these two are the nodes and this is the master node so these three got automatically provisioned by the cops command that i ran all right so this may take a while for it to get validated a good 5 to 10 minutes so what i'll do is i'll pause the video now and i'll come back once on the server the cluster is up and running it's been a good eight to nine minutes since i started my cluster so let me validate now okay seems good so there's a master node minimum one max one some nodes which are nothing but the slave nodes or the worker nodes there are two of them due to micro subnet so my clusters seem to be up and running and my cluster name is simply learn.n so what i would want to do is now let me just log into the master and see if i can run some parts so let me get back to my installation steps here the validation cluster is done so let me log into my since my cluster name is simplyland. and this is the way to log into my box so let me get into my box so if you see here uh this is the hostname of the machine that i'm currently in if you see this this is nothing but our this cop server this is the server now from here i'm trying to get into the master box all right so if you see the ip address has changed i was in a different box i'm in a different box now if i see host name you'll find a different host name so this is 153 which is nothing but the master node yep it's 153 this is the particular host so i'm getting into this machine now so i started the cluster from my cop server here it ran and brought up three nodes so i'm actually getting into my master node and see if i can run some parts on it all right so let me try kubectl uh get cluster info try cube ctl get nodes all right so there are three nodes here master node and um one master node and two worker nodes so let me see if i have some pods here cubic ctl get pods so there's no port as of now so what i'll try to do is let me just pin up a very very simple part just to check if my connections is everything is correct or not correct i have a simple node.js application that i've built uh i've got a container for that this is already pushed to the docker i've registered is called simply learn docker hub and the image name that i have here is called my node app and i will use this image to power one of my ports that i will run so this is the way in which i'm going to launch my pod let me show you these commands cube ctl run the name for my deployment that i'm going to run hyphen iphone image and this is the image that is going to be powering my container so this is simply learn docker hub forward slash my node app hyphen f1 replica is equal to i want two replicas to be running and the port that i want to expose this particular part is on eight zero eight zero so let me run this as soon as this is run it creates something called as a deployment all right now let me just say um i'm not really interested in the deployment i'm interested in checking if all the parts are up and running so i will say cube ctl get both all right so that was pretty fast so i have two parts that are running here these are two replicas because i asked for two replicas these parts are running so i can run cube ctl describe pod pickup all right i can pick up any of the pod name and see what is the pod what is the information does it contain from what is the image that is pulling what is the image image name id this is actually running my pod which is actually spinning up a container great so far so good so in kubernetes the pods can are ephemeral so they can be there at any time cannot beat any time if i need to expose my pot outside i will have to create a service for my part so what i'll do is i'll create a very very simple service by exposing this particular deployment before that let me check cube cdl get deployment so there's a deployment that is created the deployment name is simply run app so i'll expose my deployment simply learn app as uh yeah i'll just expose this let me see all right i am not specifying what type and all i don't want to get into the complications of uh what are the different types of exposing the service and all that stuff so if i don't specify anything uh it gives me something closer cluster ip so this is where my pod is actually exposed so let me just check if at all this part is up and running i'll just try a simple curl command curl http colon this is my cluster ip and uh the port is 8080 if at all i had this it's actually hitting my application and giving me uh whatever um i put a simple uh sys dot out kind of a thing where i'm just printing um the container id of whatever pod is serving uh being sold out of so i'm actually hitting my container and i'm getting this output from the content so everything looks good my content is up and running uh so let me just go and clean it up i'll say equip ctl uh delete deployment simply learn app this should get rid of all the parts that i created cube ctl getpod all right these parts are intimidating things let me just check cubectl get services there's one service i don't want this service let me delete that cubectl delete service i want this all right that sounds good so let me come back to my host my cop server from where i'm running so i managed to successfully verify this part see if everything is up and running so what i'll do is i'll just go ahead and complete this demo by going and getting rid of my cluster so corpse delete cluster hyphen f1 yes all right so this will ensure that it will clean up my complete cluster that i created so i had three or four running instances if you see that all the three are shutting down because you know the cop server which had cops installed on it uh has now got a mandate to go ahead and shut down and clean up the whole instances and all those things that are created as a part of my deployment today i'm going to tell you how you can become an aws solutions architect so who is an aw solutions architect now the main role of an aws solutions architect is to help you deploy your applications on the aws platform now this is nothing but a cloud computing platform so it's not to say that you can't deploy your applications on the cloud computing platform yourself it's just that when it comes to organizations the applications that you need to deploy become a whole lot more complex that's where an aw solutions architect can help ever since cloud computing became a thing companies around the world have started migrating their physical infrastructure onto the cloud that's what an aws solutions architect does they help you migrate your physical infrastructure onto the aws cloud companies around the world work on a budget an aws solutions architect will help design a cloud infrastructure based on the organization's budget before that can be done however an aw solutions architect has to create a design with an intricate and detailed blueprint of the cloud infrastructure that they plan to set up now aw solutions architects also have to focus mainly on non-functional requirements like usability reliability scalability and performance of the cloud infrastructure they're also responsible when it comes to minimizing risks that an organization can face when it comes to cloud computing platforms now they could face risks like security leaks calculation mistakes and application down times an aw solutions architect has to ensure that these don't happen before we can talk about how you can become an aws solutions architect i have some exciting news for you guys we've launched our own youtube community you'll get to see a lot of quizzes polls offers and much more to make your learning experience a whole lot more fun you can find all of this on your subscription feed or you can also click on the top right corner right now to get started and let's get back to how can you become an aws solutions architect now to become an aw certified solutions architect you need to clear the aw certified solutions architect associate level examination now here's some details about it the exam score ranges from 100 to 1000 marks and the minimum passing score is 720. however there's a catch the passing marks are actually set using statistical analysis so they can be changed based on how difficult the examination actually is the exam fee is 150 and you can also take a practice examination which costs 20 dollars now regardless of the examination you take be it solutions architect sysops administrator or developer any associate level examination costs hundred and fifty dollars for the professional level examination it's three hundred dollars now if you want to learn more about the aws certifications i suggest you click on the top right corner and watch our video aw certifications in 10 minutes to learn more now the exam duration is of 130 minutes and you have two types of questions multiple choice and multiple answer now the multiple choice questions have four options on which one of them is right and you have multiple answer where you have five options out of which two of them are correct you can take the examination in english japanese korean and simplified chinese now let's talk about how you can schedule an examination first let's go to google and search for aws certifications click on the first link so on this page you can go to the bottom and find the different certifications aws provides click on architect and select the awa certified solutions architect associate certification click on register now and here you need to click on the aw certification account you can sign in with your amazon account or create a new one i already have an amazon account that i'll be using for signing in here in case you don't you can click on the create your amazon account button here and create an account for yourself now after that is done you can schedule a new examination and you can scroll down to the bottom here aw certified solutions architect associate level certification click on schedule exam press continue and here you need to select your language which i'm assuming is english i'm from india so i don't need to change anything here but you can change your country your time zone and other details based on your requirement and select your preferred month i want to do it in october now select search for exam center select the one that's closest to you i'm going to select this one and here you can select the day that you want to take your test and the time available i want to do it on 15 select the time that i want and press continue now you can go through all the details here change them if you want to otherwise you can press continue after this is done close and here we have the final step which is the fees you can enter your details here and pay now to finish the process now let's look at an outline of the exam content now what you're going to see in the exam are five major domains and here you have each of the domains with their respective weightages first you have at 34 designing resilient architectures at 24 percent you have both defining performant architectures and specifying secure applications and architectures at ten percent you have designing cost optimized architectures and at six percent you have defining operationally excellent architectures now let's look at each of these domains in detail now the first domain or design resilient architectures can be divided into four parts firstly you need to know how you can choose reliable or resilient storage using services like aws s3 aws glacier and aws ebs then you have how you can design decoupling mechanisms using aws services now this is possible with the help of aws sns now these aren't the only services that enable this these are just some of the services then how you can design a multi-tier architecture solution now this is important because you need to know how you can create a solution that involves several other services then you need to know how you can design highly available and fault tolerant architectures now for the second domain defining performance architectures first you have how to choose performance storages and databases services that are used are aws rds aws redshift and aws dynamodb the second step is how you can apply caching to improve performance a service that can be used for this is aws elastic cache third how you can design solutions with elasticity and scalability you have aws lambda aws cloud watch and aws data pipeline now for the third domain which is specifying secure applications and architectures you need to know how you can secure applications using services like aws inspector aws cloudtrail and aws iam you need to know how to secure data using cloud hsm and aws macy and how you can define the networking infrastructure using cloudfront vpc and elastic load balancer for the fourth domain you have designing cost optimized architectures firstly you need to know how you can design cost optimized compute solutions you can use aws ec2 elastic bean stock lambda and aws light sale then you need to know how you can design cost optimized storage solutions using aws s3 glacier ebs and elastic file system and the final domain to define operationally excellent architectures you need to set up design features and solutions that enable operational excellence now some of the features are that you perform operations as code you annotate documentation you make frequent and small reversible changes and anticipate and tackle failures now let's have a look at the job opportunities when it comes to aw solutions architects now if you have a look at this graph you can see that aws has always provided a lot more job postings as compared to the other two giants in the cloud computing domain which are microsoft azure and google cloud platform now if you do a little bit of research you'll find out that there's a significant lacking of experienced personnel when it comes to aws so i would suggest this is the best time for you to get certified in aws so how can simply learn help you with your aws certifications now we are on the sampler on home page and here we are the adw solutions architect course on this course which is the aws solutions architect certification training course we have 36 hours of instructor-led training 20 hours of self-paced learning 16 live demos three simulation exams three hands-on practice projects and so much more here our main emphasis is on making sure that you have enough hands-on experience with the services that you crack the examination at your first attempt so here we also go through some of the more important services that you need to learn like iam vpc ec2 s3 and so much more we'll also talk about databases application services security practices disaster recovery and so on we even have practice examinations to help you prepare for the real one choose from over 300 in-demand skills and get access to 1 000 plus hours of video content for free visit skill up by simply learn click on the link in the description to know more i'm here to walk you through some of the aws interview questions which we find are important and our hope is that you would use this material in your interview preparation and be able to crack that cloud interview and step into your dream cloud job by the way i'm and cloud technical architect trainer and an interview panelist for cloud network and devops so as you progress in watching you're going to see that these questions are practical scenario based questions that tests the depth of the knowledge of a person in a particular aws product or in a particular aws architecture so why wait let's move on all right so in an interview you would find yourself with a question that might ask you define and explain the three basic types of cloud services and the aws products that are built based on them see here it's a very straightforward question just explain three basic types of cloud service and when we talk about basic type of cloud service it's compute obviously that's a very basic service storage obviously because you need to store your data somewhere and networking that actually connects a couple of other services to your application these basic will not include monitoring these basic will not include analytics because they are considered as optional they are considered as advanced services you could choose a non-cloud service or a product for monitoring of and for analytics so they're not considered as basic so when we talk about basics they are compute storage and networking and the second part of the question says explain some of the aws products that are built based on them of course compute ec2 is a major one that's that's the major share of the compute resource and then we have platform as a service which is elastic bean stock and then function as a service which is lambda auto scaling and light cell are also part of compute services so the compute domain it really helps us to run any application and the compute service helps us in managing the scaling and deployment of an application again lambda is a compute server so the compute service also helps in running event initiated stateless applications the next one was storage a lot of emphasis is on storage these days because if there's one thing that grows in a network on a daily basis that storage every new day we have new data to store process manage so storage is again a basic and an important cloud service and the products that are built based on the storage services are s3 object storage glacier for archiving ebs elastic block storage as a drive attachment for the ec2 instances and the efs file share for the ec2 instances so the storage domain helps in the following aspects it holds all the information that the application uses so it's the application data and we can also archive old data using storage which would be glacier and any object files and any requirement for block storage can be met through elastic block store and s3 which is again an object storage talking about networks it's just not important to answer the question with the name of the services and the name of the product it'll also be good if you could go in depth and explain how they can be used right so that actually proves you to be a person knowledgeable enough in that particular service or product so talking about networking domain vpc networking can't imagine networking without vpc in in the cloud environment especially in aws cloud environment and then we have route 53 for domain resolution or for dns and then we have cloudfront which is an edge caching service that helps customers get our customers to read their application with low latency so networking domain helps with some of the following use cases it controls and manages the connectivity of the aws services within our account and we can also pick an ip address range if you're a network engineer or if you are somebody who works in networks or planning to work a network you will soon realize the importance of choosing your own ip address for easy remembering so having an option to have your own ip address in the cloud owned range of ip address in the cloud it really helps really really helps in cloud networking the other question that get asked would be the difference between the availability zone and the region actually the question generally gets asked so to test how well you can actually differentiate and also correlate the availability zone and the region relationship right so a region is a separate geographic area like the us2s1 i mean which represents uh north california or the ap south which represents mumbai so regions are a separate geographic area on the contrary availability zone resides inside the region you shouldn't stop there you should go further and explain about availability zones and availability zones are isolated from each other and some of the services will replicate themselves within the availability zone so availability zone does replication within them but regions they don't generally do replication between them the other question you could be asked is what is auto scaling what do we achieve by auto scaling so in short auto scaling it helps us to automatically provision and launch new instances whenever there is an demand it not only helps us meeting the increasing demand it also helps in reducing the resource usage when there is low demand so auto scaling also allows us to decrease the resources or resource capacity as per the need of that particular r now this helps business in not worrying about putting more effort in managing or continuously monitoring the server to see if they have the needed resource or not because auto scaling is going to handle it for us so business does not need to worry about it and auto scaling is one big reason why people would want to go and pick a cloud service especially an aws service the ability to increase and shrink based on the need of that r that's how powerful is auto scaling the other question you could get asked is what's due targeting in cloud front now we know that cloudfront is caching and it caches content globally in the amazon caching servers global wide the whole point is to provide users worldwide access to the data from a very nearest server possible that's the whole point in using or going for cloudfront then what do you mean by geotargeting geotargeting is showing customer and specific content based on language we can customize the content based on what's popular in that place we can actually customize the content the url is the same but we could actually change the content a little bit not the whole content otherwise it would be dynamic but we can change the content a little bit a specific a file or a picture or a particular link in a website and show customized content to users who will be in different parts of the globe so how does it happen cloudfront will detect the country where the viewers are located and it will forward the country code to the origin server and once the origin server gets the specialized or specific country code it will change the content and it will send to the caching server and it get cached there forever and the user gets to view a content which is personalized for them for the country they are in the other question you could get asked is the steps involved in using cloud formation or creating a cloud formation or backing up an environment within cloud formation template we all know that if there is a template we can simply run it and it provisions the environment but there is a lot more going into it so the first step in moving towards infrastructure as a code is to create the cloud formation template which as of now supports json and yaml file format so first create the cloud formation template and then save the code in an x3 bucket s3 bucket serves as the repository for our code and then from the cloud formation call the file in the s3 bucket and create a stack and now cloud formation uses the file reads the file understands services that are being called understands the order understands how they are connected with each other cloud formation is actually an intelligent service it understands the relation based on the code it would understand the relationship between the different services and it would set an order for itself and then would provision the services one after the other let's say a service has a dependency and the dependent service the other service which this service let's say service a and b service b is dependent on service a let's say cloud formation is an intelligent service it would provision the resource a first and then would provision resource b what happens if we inverse the order if we inverse the order resource b first gets provision and because it does not have dependency changes that the cloud formation's default behavior is that if something is not provisioned properly something is not healthy it could roll back chances that the environment provisioning will roll back so to avoid that cloud formation first provisions all the services that has or that's dependent on that's depended by another service so it provisions those service first and then provisions the services that has dependencies and if you're being hired for a devops or you know if the interview wanted to test your skill on systems side this definitely would be a question in his list how do you upgrade or downgrade a system with near zero downtime now everybody's moving towards zero downtime or near zero downtime all of them want their application to be highly available so the question would be how do you actually upgrade or downgrade a system with near zero downtime now we all know that i can upgrade an ec2 instance to a better ec2 instance by changing the instance type stopping and starting but stopping and starting is going to cause a downtime right so that's you should be answering or you shouldn't be thinking in those terms because that's the wrong answer specifically the interviewer wants to know how do you upgrade a system with zero downtime so upgrading system with zero downtime it includes launching another system parallelly with the bigger ec2 instance type over the bigger capacity and install all that's needed if you are going to use an ami of the old machine well and good you don't have to go through installing all the updates and installing all the application from the ami once you have launched it in a bigger instance locally test the application to see if it is working don't put it on production yet test the application to see if it is working and if the application works we can actually swap if your server is behind and behind route 53 let's say all that you could do is go to rot 53 update the information with the new ip address new ip address of the new server and that's going to send traffic to the new server now so the cutover is handled or if you're using static ip you can actually remove the static ip from the old machine and assign it to the new machine that's one way of doing it or if you are using elastic nic card you can actually remove the new card from the old machine and attach the new card to the new machine so that way we would get near zero downtime if you're hired for an architect level you should be worrying about cost as well along with the technology and this question would test how well you manage cost so what are the tools and techniques we can use in aws to identify and correct identify and know that we are paying the correct amount for the resources that we are using or how do you get a visibility of your aws resources running one way is to check the billing there is a place where you can check the top services that were utilized it could be free and it could be paid service as well top services that can be utilized it's actually in the dashboard of the cost management console so that table here shows the top five most used services so looking at it you can get it all right so i'm using a lot of storage i'm using a lot of ec2 why is storage high you can go and try to justify that and you will find if you are storing things that should be storing and then clean it up why is compute capacity so high why is data transfer so high so if you start thinking in those levels you'll be able to dig in and clean up unnecessary and be able to save your bill and there are cost explorer services available which will help you to view your usage pattern or view your spending for the past 13 months or so and it will also forecast for the next three months now how much will you be using if your patent is like this that will actually help and will give you a visibility on how much you have spent how much you will be spending if the trend continues budgets are another excellent a way to control cost you can actually set up budget alright this is how much i am willing to spend for this application for this team or for this month for this particular resource so you can actually put a budget mark and anytime it exceeds any time it's nearing you would get an alarm saying that well we're about to reach the allocated budget amount stuff like that that way you can go back and know and you know that how much the bill is going to be for that month or you can take steps to control bill amount for that particular month so aws budget is another very good tool that you could use cost allocation tags helps in identifying which team or which resource has spent more in that particular month instead of looking at the bill as one list with no specifications into it and looking at it as an expenditure list you can actually break it down and tag the expenditure to the teams with cost allocation tax the dev team has spent so much the uh production team has spent so much the training team has spent more than the dev and the production team why is that now you'll be able to you know think in those levels only if you have cost allocation tax now cost allocation tags are nothing but the tags that you would put when you create a resource so for production services you would put as a production tab you would create a production tag and you would associate that resources to it and at a later point when you actually pull up your bill that's going to show a detailed a list of this is the owner this is the group and this is how much they have used in the last month and you can move forward with your investigation and encourage or stop users using more services with the cost allocation tax the other famous question is are there any other tools or is there any other way of accessing aws resource other than the console console is gui right so in other words other than gui how would you use the aws resource and how familiar are you with those tools and technologies the other tools that are available that we can leverage and access the aws resource are of course hoodie you can configure puri to access the aws resources like log into an ec2 instance and ec2 instance does not always have to be logged in through the console you could use putty to log into an ec2 instance like the jump box like the proxy machine and like the gateway machine and from there you can actually access the rest of the resources so this is an alternative to the console and of course we have the aws cli in any of the linux machines or windows machines we can install so that's point two three and four we can install aws cli for linux windows also for mac so we can install them and from there from your local machine we can access run aws commands and access provision monitor the aws resources the other ones are we can access the aws resource programmatically using aws sdk and eclipse so these are a bunch of options we have to use the aws resource other than the console if you're interviewed in a company or by a company that focuses more on security and want to use aws native services for their security then you would come across this question what services can be used to create a centralized logging solution the basic services we could use our cloud watch logs store them in s3 and then use elasticsearch to visualize them and use kinesis to move the data from s3 to elasticsearch right so log management it actually helps organizations to track the relationship between operational and security changes and the events that got triggered based on those logs instead of logging into an instance or instead of logging into the environment and checking the resources physically i can come to a fair conclusion by just looking at the logs every time there's a change the system will scream and it gets tracked in the cloud watch and then cloud watch pushes it to s3 kinesis pushes the data from s3 to elasticsearch and i can do a time-based filter and i would get an a fair understanding of what was going on in the environment for the past one or whatever the time window that i wanted to look at so it helps in getting a good understanding of the infrastructure as a whole all the logs are getting saved in one place so all the infrastructure logs are getting saved in one place so it's easy for me to look at it in an infrastructure perspective so we know the services that can be used and here are some of the services and how they actually connect to each other it could be logs that belongs to a one account it could be logs that belongs to multiple accounts it doesn't matter you know those three services are gonna work fairly good and they're gonna inject or they're gonna like suck logs from the other accounts put it in one place and help us to monitor so as you see you have cloud watch here that actually tracks the metrics you can also use cloud trail if you want to log api calls as well push them in an s3 bucket so there are different types of block flow logs are getting captured in an instance application logs are getting captured from the same vpc from a different vpc from the same account from a different account and all of them are analyzed using elasticsearch using the kibana client so step one is to deploy the ecs cluster step two is to restrict access to the ecs cluster because it's valid data you don't want anybody to put their hands and access their data so resting access to the ecs dashboard and we could use lambda also to push the uh data from cloudwatch to the elasticsearch domain and then kibana is actually the graphical tool that helps us to visualize the logs instead of looking at log as just statements or in a bunch of characters a bunch of files kibana helps us to analyze the logs in a graphical or a chart or a bar diagram format again in an interview the interview is more concerned about testing your knowledge on aw security products especially on the logging monitoring even management or incident management then you could have a question like this what are the native aws security logging capabilities now most of the services have their own logging in them like have their own logging like s3 s3 has its own login and cloudfront has its own logging ds has its own logging vpc has its own logging in additional there are account level logins like a cloudtrail and aws config services so there are variety of logging options available in the aws like cloud soil config cloudfront redshift logging rds logging vpc flow logs s3 object logging s3 access logging stuff like that so we're going to look at two servers in specific cloud trail now this cloud trail the very first product in that picture we just stopped the cloud trail provides an very high level history of the api calls for all the account and with that we can actually perform a very good security analysis a security analysis of our account and these logs are actually delivered to you can configure it they can be delivered to s3 for long time archivals and based on a particular event it can also send an email notification to us saying hey just got this error thought i'll let you know stuff like that the other one is config service a config service helps us to understand the configuration changes that happened in our environment and we can also set up notifications based on the configuration changes so it records the cumulative changes that are made in a short period of time so if you want to go through the lifetime of a particular resource what are the things that happened what are the things it went through they can be looked at using aws config all right the other question you could get asked is if you know your role includes taking care of cloud security as well then the other question you could get asked is the native services that amazon provides or to mitigate ddos which is denial of service now not all companies would go with amazon native services but there are some companies which want to stick with amazon native services just to save them from the headache of managing the other softwares or bringing in another tool a third-party tool into managing ddos they simply want to stick with amazon proprietary amazon native services and a lot of companies are using amazon service to prevent ddos denial of service now denial of service is if you already know what denial of service is well and good if you do not know then let's know it now denial of service is a user trying to or maliciously making attempt to access a website or an application the user would actually create multiple sessions and he would occupy all the sessions and he would not let legitimate users access the servers so he's in turn denying the service for the user a quick picture review of what denial of services now look at it these users instead of making one connection they are making multiple connections and there are cheap software programs available that would actually trigger connections from different computers in the internet with different mac addresses so everything kind of looks legitimate for the server and it would accept those connections and it would keep the sessions open the actual users won't be able to use them so that's denying the service for the actual users denial of service all right and distributed denial of service is generating attacks from multiple places you know from a distributed environment so that's distributed denial of service so the tools the native tools that helps us to prevent the denial of service attacks in aws is cloud shield and web access firewall aw swap now they are the major ones they are designed to mitigate a denial of service if your website is often bothered by denial of service then we should be using aws shield or aws vaf and there are a couple of other tools that also does when i say that also does denial of service is not their primary job but you could use them for denial of service route 53's purpose is to provide dns cloudfront is to provide caching elastic load balancer elbs work is to provide load balancing vpc is to create and secure a virtual private environment but they also support mitigating denial of service but not to the extent you would get in aws shield and aws web so aws shield and waff are the primary ones but the rest can also be used to mitigate distributed denial of service the other tricky question is this actually will test your familiarity with the region and the services available in the region so when you're trying to provision a service in a particular region you're not seeing the service in that region how do we go about fixing it or how do we go about using the service in the cloud it's a tricky question and if you have not gone through such situation you can totally blow it away you really need to have a good understanding on regions the services available in those regions and what if a particular service is not available how to go about doing it the answer is not all services are available in all regions anytime amazon announces a new service they don't immediately publish them on all regions they start small and as in when the traffic increases as and when it becomes more likeable to the customers they actually move the service to different regions so as you see in this picture within america not virginia has more services compared to ohio or compared to north california so within not america itself north virginia is the preferred one so similarly there are preferred regions within europe middle east and africa and preferred regions within asia pacific so anytime we don't see a service in a particular region chances that the service is not available in that region yet we got to check the documentation and find the nearest region that offers that service and start using the service from that region now you might think well if i'm looking for a service in asia let's say in mumbai and if it is not available why not simply switch to north virginia and start using it you could but you know that's going to add more latency to your application so that's why we need to check for application which is check for region which is very near to the place where you want to serve your customers and find nearest region instead of always going back to north virginia and deploying an application in north virginia again there's a place there's a link in aws.com that you can go and look for services available in different region and that's exactly what you're seeing here and if your service is not available in a particular region switch to the other region that provides your service the nearest other region that provides that service and start using service from there with the uh coming up of cloud a lot of companies have turned down their monitoring team instead they want to go with the monitorings that cloud provides you know nobody wants to or at least many people don't want to go through the hassle of at least new startups and new companies that are thinking of having a monitoring environment that they don't want to go with traditional knock monitoring instead they would like to leverage aws monitorings available because it monitors a lot of stuff not just the availability but it monitors a lot of stuff like failures errors it also triggers emails stuff like that so how do you actually set up a monitor to website how to set up a monitor to monitor the website metrics in real time in aws the simple way anytime you have a question about monitoring cloudwatch should strike your mind because cloudwatch is meant for monitoring is meant for collecting metrics is meant for providing graphical representation of what's going on in a particular network at a particular point of time so cloudwatch cloudwatch helps us to monitor applications and using cloudwatch we can monitor the state changes not only the state changes the auto scaling life cycle events anytime there are there are more services added there is a reduction in the number of servers because of less usage a very informative messages can be received through cloud watch any cloud watch can now support scheduled events if you want to schedule anything cloudwatch has an event that would schedule an action right schedule a trigger time based not incident based you know anything happening and then you get an action happening that's incident based on the other hand you can simply schedule few things on time based so that's possible with cloud watch so this cloud watch integrates very well with a lot of other services like notifications for notifying the user or for notifying the administrator about it and it can integrate well with lambda so to trigger an action anytime you are designing an auto healing environment this cloud watch can actually monitor and send an email if we are integrating it with sns simple notification service or this cloud watch can monitor and based on what's happening it can trigger an event in lambda and that would in turn run a function till the environment comes back to normal so cloudwatch integrates well with a lot of other aw services all right so cloudwatch has uh three statuses green when everything is going good yellow when the service is degraded and red when the service is not available green is good so we don't have to do anything about it but anytime there's an ello the picture that we're looking at it's actually calling and lambda function to debug the application and to fix it and anytime there's a red alert it immediately notifies the owner of the application about well the service is down and here is the report that i have here is the metrics that i've collected about the service stuff like that if the job role requires you to manage the servers as well there are certain job roles which are on the system side there are certain job roles which is development plus system side now you're responsible for the application and the server as well so if that's the case you might be tested with some basic questions like the different types of virtualization and aws and what are the difference between them all right the three major types of virtualization are hvm which is hardware virtual machine the other one is pv para virtualization and the third one is pv on hvm para virtualization on hardware virtual module all right the difference between them or actually describing them is actually the difference between them hvm it's actually a fully virtualized hardware you know the whole hardware is virtualized and all virtual machines act separate from each other and these vms are booted by executing master boot record in the root block and when we talk about para virtualization paragraph is actually the special boot loader which boots the pv amis and when we talk about pv on hvm it's it's actually the marriage between hvm and pv and this para virtualization on hvm in other words pv on hvm it actually helps operating system take advantage in storage and the network input output available through the host another good question is name some of the services that are not region specific now you've been thought that all services are within a region and some services are within an availability zone for example ec2 is within an availability zone ebs is within an availability zone s3 is region specific dynamodb is region specific stuff like that vpc is both availability and region specific meaning you know subnets are availability zone specific and vpcs region specific stuff like that so you might have thought you might have learned in that combination but there could be some tricky questions that tests you how well you have understood the region non-region and availability non-availability services i should say there are services that are not region specific that would be iam so we can't have im for every availability zone and for every region which means you know users will have to use one username and password for one region and anytime they switch to another region they will have to use another username and password that that's more work and that's not a good design as well authentication has to be global so im is a global service and which means its not region specific on the other hand route 53 is again a regional specific so we cannot have route 53 for every region route 53 is not a region specific service it's a global service and it's one application users access from everywhere or from every part of the world so we can't have one url or one dns name for each region if your application is a global application and then web application firewall works well with cloudfront then cloudfront is a region based service so the web application firewall it's not region specific service it's a global service and cloudfront is again a global service though you can you know cache content on a continent and country basis it's still considered a global service right it's not bound to any region so when you activate cloud front you're activating it away from region or availability zone so when you're activating a web application firewall because it's not a region-specific service you're activating it away from availability zone and regions so a quick recap i am users groups roles and accounts they are global services they can be used globally all 53 services are offered at edge locations and they are global as well web application firewall a service that protects our web application from common web exploits they are global service as well cloudfront cloudfront is global content delivery network cdn and they are offered at edge locations which are a global service in other words non-region specific service or beyond region service all right this is another good question as well in the project that you are being interviewed if they really want to secure their environment using nat or if they are already securing their environment using nat by any of these two methods like nat gateway or nat instances you can expect this question what are the difference between and that gateway and that instances now they both saw the same thing right so they're not two different services trying to achieve two different things they both serve the same thing but still they do have differences in them right on a high level they both achieve providing netting for the service behind it but the difference comes when we talk about the availability of it that gateway is a managed service by amazon whereas nat instance is managed by us now i'm talking about the third point maintenance here nat gateway is managed by amazon that instance is managed by us and availability of nat gateway is very high and availability of that instance is less compared to the nat gateway because it's managed by us you know it's on an ec2 instance which could actually fail and if it fails we'll have to relaunch it but if it is not gateway if something happens to that service amazon would take care of reprovisioning it and talking about bandwidth it can burst up to 75 gigabits now traffic through the net gateway can burst up to 75 gigabits but for that instance it actually depends on the server that we launched and if we are launching a t2 micro it barely gets any bandwidth so there's a difference there and the performance because it's highly available because of the bigger pipe 75 gigabits now the performance of the nat gateway is very high but the performance of the nat instance is going to be average again it depends on the size of the net instance that we pick and billing a billing for nat gateway is the number of gateways that we provision and the duration for which we use the nat gateway but billing for that instance is number of instance and the type of instance that we use of course number of instance duration and the type of instance that we use security in that gateway cannot be uh assigned meaning it already comes with full packed security but in that instance security is a bit customizable i can go and change the security because it's a server managed by me or managed by us i can always change the security well allow this allow don't allow this stuff like that size and load of the night gateway is uniform but the size and the load of the uh that instance changes as per an gateway is a fixed product but not instance can be small instance can be a big instance so the size and the load through it varies right the other question you could get asked is what are the difference between stopping and terminating an ec2 instance now you will be able to answer only if you have worked on environments where you have your instance stopped and where you have your instance terminated if you have only used lab and are attending the interview chances are that you might you always lost when answering this question it might look like both are the same well stopping and terminating both are the same but there is a difference in it so when you stop an instance it actually performs a normal shutdown on the instance and it simply moves the instance to the stopped state but when you actually terminate the instance the instance is moved to this stop state the ebs volumes that are attached to it are deleted and removed and we'll never be able to recover them again so that's a big difference between stopping and terminating an instance if you're thinking of using the instance again along with the data in it you should only be thinking of stopping the instance but you should be terminating the instance only if you want to get rid of that instance forever if you are being interviewed for an architect level position or a junior architect level position or even a cloud consultant level position or even in an engineering position this is a very common question that gets asked what are the different types of easy to instances based on their cost or based on how we pay them right they're all compute capacity for example the different types are on demand instances spot instances and reserved instances it kind of looks the same they all provide the compute capacity they all provide the same type of hardwares for us but if you are looking at cost saving or optimizing cost in our environment we got to be very careful about which one are we picking now we might think that well i'll go with demand instance because i pay on a per hour basis which is cheap you know i can use them anytime i want and anytime i don't want i can simply get rid of it by terminating it you're right but if the requirement is to use the service for one year the requirement is to use the service for three years then you'll be wasting a lot of money buying on-demand instances you'll be wasting a lot of money paying on an hourly basis instead we should be going for reserved instance where we can reserve the capacity for the complete one year or complete three years and save huge amount in buying reserved instances all right so under man is cheap to start with if you're only planning to use it for a short while but if you're planning to run it for a long while then we should be going for reserved instance that is what is cost efficient so spot instance is cheaper than on-demand instance and there are different use cases for spot instance as well so let's look at one after the other the on-demand instance the on-demand instance is purchased at a fixed rate per r this is very short-term and irregular workloads and for testing for development on demand instance is a very good use case we should be using on demand for production spot instance spot instance allows users to purchase ec2 at a reduced price and anytime we have more instances we can always go and sell it in spot instances i'm referring to anytime we have more reserved instances we can always sell them in spot instance catalog and the way we buy spot instance is we actually put a budget this is how much i'm willing to pay all right would you be able to give service within this cost so anytime the price comes down and meets the cost that we have put in will be assigned an instance and anytime the price shoots up the instance will be taken away from us but in case of on demand instances we have bought that instance for that particular r and it stays with us but with spot instances it varies based on the price if you meet the price you get the instance if you don't meet the price goes away to somebody else and the spot instance availability is actually based on supply and demand in the market there's no guarantee that you will get spot instance at all time all right so that's a caveat there you should be familiar with that's a caveat that you should be aware when you are proposing somebody that we can go for spot instance and save money it's not always going to be available if you want your spot instance to be available to you then we need to carefully watch the history of the price of the spot instance now how much was it last month and how much was it how much is it this month so how can i code or how much can i code stuff like that so you got to look at those history before you propose somebody that well we're going to save money using spot instance on the other hand reserved instance provide cost savings for the company we can opt for reserved instances for you know one year or three years there are actually three types of reserved instances light medium and heavy reserved instances they are based on the amount that we would be paying and cost benefit also depends with reserved instance the cost benefit also depends based on are we doing all upfront or no upfront or partial payment then split the rest as monthly payments so there are many purchase options available but overall if you're looking at using an application for the next one year and three years you should not be going for on-demand instance you should be going for reserved instance and that's what gives you the cost benefit and in an aws interview sometimes you might be asked you know how you interact with the aws environment are you using cli are you using console and depending on your answer whether console or a cli the panelist put a score okay this person is cli specific this person is console specific or this person has used aws environment through the sdk stuff like that so this question tests whether you are a cli person or an console person and the question goes like this how do you set up ssh agent forwarding so that you do not have to copy the key every time you log in if you have used puri anytime if you want to log into an ec2 instance you will have to put the ip and the port number along with that you will have to map or we will have to map the key in the puri and this has to be done every time that's what we would have done in our lab environments right but in production environment using the same key or mapping the same key again and again every time it's actually an hassle it's considered as a blocker so you might want to cache it you might want to permanently add it in your putty session so you can immediately log in and start using it so here in the place where you would actually map the private key there's a quick button that actually fixes or that actually binds your ssh to your instance so we can enable ssh agent forwarding that will actually bind our key to the ssh and next time when we try to log in you don't have to always go through mapping the key and trying to log in all right this question what are solaris and ax operating systems are they available with aws that question generally gets asked to test how familiar are you with the amis available how familiar are you with ec2 how familiar are you with the ec2 hardwares available that basically test that now the first question or the first thought that comes to your mind is well everything is available with aws i've seen windows i've seen ubuntu i've seen red hat i've seen amazon amis and if i don't see my operating system there i can always go to marketplace and try them if i don't find a marketplace i can always go to community and try them so a lot of ami is available there a lot of operating systems available i will be able to find solaris and ax but that's not the case solar s and ax are not available with aws that's because solaris uses a different and solaris does not support the architecture does not support public cloud currently the same goes for ax as well and they run on power cpu and not on intel and as of now amazon does not provide power machines this should not be confused with the hpc which is a high performance computing should not be confused with that now these are different hardwares different cpu itself that the cloud providers they do not provide yet another question you could get asked in organizations that would want to automate their infrastructure using amazon native services would be how do you actually recover an ec2 instance or auto recover an ec2 instance when it fails well we know that ec2 instances are considered as immutable meaning irreparable we don't spend time fixing bugs in an os stuff like that you know once an ec2 instance crashes like it goes on a always panic or there are various reasons why it would fail so we don't have to really worry about fixing it we can always relaunch that instance and that would fix it but what if it happens at two o'clock in the night what if it happens that during a weekend when nobody's in office looking or monitoring those instances so you would want to automate that not only on a weekend or during midnight but it's general practice good to automate it so you could face this question how do you actually automate an ec2 instance once it fails and the answer to that question is using cloudwatch we can recover the instance so as you see there is an alarm threshold a set in cloud watch and once the threshold is met meaning if there is an error if there is a failure if the ec2 instance is not responding for a certain while we can set an alarm and once the alarm is met let's say the cpu utilization stayed high for five minutes all right it's not taking any new connections or the instance is not pinging for five minutes or in this case it's two minutes it's not pinging so it's not going to respond connection so in those cases you would want to automatically recover that ec2 instance by rebooting the instance all right now look at this the take this action section under the action so there we have a bunch of options like recover this instance meaning reboot the instance so that's how we would recover the other two options are beyond the scope of the question but still you can go ahead and apply just like i'm going to do it so the other option is stop the instance that's very useful when you want to stop instances that are having low utilizations nobody's using the system as of now you don't want them to be running and wasting the cloud expenditure so you can actually set an alarm that stops the ec2 instance that's having low utilization so somebody was working in an instance and they left it without or they forgot to shut down that instance and it gets i mean they will only use it again the next day morning so in between there could be like 12 hours that the system is running idle nobody's using it and you're paying for it so you can identify such instances and actually stop them when the cpu utilization is low meaning nobody is using it the other one is to terminate let's say you want to give system to somebody temporarily and you don't want them to hand the system back to you right this is actually an idea in other words this is actually the scenario so you hand over a system to somebody and when they're done they're done we can actually terminate the system so you could instruct the other person to terminate the system when they're done and they could forget then the instance could be running forever or you can monitor the system after the specified time is over and you can terminate the system or best part you can automate the system termination so you assign a system to somebody and then turn on this cloud watch action to terminate the instance when the cpu is low for like two hours meaning they've already left or cpu is low for 30 minutes meaning they've already left stuff like that so that's possible and if you're getting hired for an system side architect or even on the sysop side you could face this question what are the common and different types of ami designs there are a lot of ami designs the question is the common ones and the difference between them so the common ones are the full back a mice and the other one is just enough os ami j e os ami and the other one is hybrid type amis so let's look at the difference between them the full backed ami just like the name says it's fully baked it's uh ready to use ami and this is the simplest ami to deploy can be a bit expensive it can be a bit cumbersome because you'll have to do a lot of work beforehand you could use the a mice a lot of planning a lot of thought process will go into it and the ami is ready to use right you hand over the ami to somebody and it's ready to use or if you want to reuse the ami it's already ready for you to use so that's full baked ami the other one is just enough operating system ami just like the name says it has i mean as you can also see in the diagram or in the picture it covers a part of the os all bootstraps are already packed properly and the security monitoring logging and the other stuff are configured at the time of deployment or at the time you would be using it so not much thought process will go in here the only focus is on choosing the operating system and what goes the operating system specific agents or bootstraps that goes into the operating system that's all we worry about the advantage of this is it's flexible meaning you can choose to install additional softwares at the time of deploying but that's going to require an additional expertise on the person who will be using the ami so that's another overhead there but the advantage is that it's kind of flexible i can change the configurations during the time of deployment the other one is a hybrid ami now the hybrid ami actually falls in between the fully baked ami and just enough operating system options so these amis have some features of the big type and some features of the just enough os type so as you see the security monitoring logging are packed in that ami and the runtime environments are installed during the time of a deployment so this is where the strict company policies would go into the ami company policies like you gotta log this you gotta monitor this these are the ports that generally gets open in all the systems stuff like that so they strictly go into the ami and sits in an ami format and during deployment you have the flexibility of choosing the different runtime and the application that sits in an ec2 instance another very famous question you would face in an interview is how can you recover login to an ec2 instance to which you lost the key well we know that if the key is lost we can't recover it there are some organizations that integrate their ec2 instances with an 80 that's different all right so you can go and reset the password and the 80 and you will be able to log into the new password but here the specific tricky question is you are using a key to log in and how do you recover if you have lost the key generally companies would have made a backup of the key so we can pick from the backup but here the specific question is we have lost the key literally no backups on the key at all so how can we log in and we know that we can't log into the instance without the key present with us so the steps to recover is that make the instance use another key and use that key to log in once the key is lost it's lost forever we won't be able to recover it you can't raise the ticket with amazon not possible they're not going to help it's beyond the scope so make the instance use another key it's only the key that's the problem you still have valid data in it you got to recover the data it's just the key that's having the problem so we can actually focus on the key part alone and change the key and that will allow us to log in so how do we do it step by step procedure so first verify the ec2 config service is running in that instance if you want you can actually beforehand install the easy to config in that service or you can actually make the easy to config run through the console just a couple of button clicks and that will make the easy to configure run in that ec2 instance and then detach the root volume for that instance of course it's going to require a stop and start to detach the root volume from the instance attach the root volume to another instance as a temporary volume or it could be a temporary instance that you've launched only to fix this issue and then login to that instance and to that particular volume and modify the configuration file configuration file modify it to use the new key and then move the root volume back to its original position and restart the instance and now the instance is going to have the new key and you also have the new key with which you can log in so that's how we go ahead and fix it now let's move on to some product specific or s3 product specific questions a general perception is s3 and ebs can be used interchangeably and the interviewer would want to test your knowledge on s3 and evs well ebs uses s3 that's true but they can't be interchangeably used so you might face this question what are some key differences between aws s3 and ebs well the differences are s3 is an object store meaning you can't install anything in it you can store drive files but you can't actually install in it it's not a file system but ebs is a file system you can install services i mean install applications in it and that's going to run stuff like that and talking about performance s3 is much faster and ebs is super faster when accessing from the instance because from the instance if you need to access s3 you will actually have to go out through the internet and access the s3 or s3 is an external server it's a very external service you'll have to go through or you'll have to go outside of your vpc to access s3 s3 does not come under a vpc but ebs comes under a vpc it's on the same vpc so you would be able to use it kind of locally compared to s3 ebs is very local so that way it's going to be faster and redundancy talking about redundancy of s3 and ebs s3 is replicated the data in s3 is replicated across the data centers but ebs is replicated within the data center meaning s3 is replicated across availability zones ebs is within an availability zone so that way the redundancy is a bit less in ebs in other words redundancy is higher in s3 than eps and talking about security of s3 is3 can be made private as well as public meaning anybody can access s3 from anywhere in the internet that's possible with s3 but ebs can only only be accessed when attached to an ec2 instance right just one instance can access it whereas s3 is publicly directly accessible the other question related to s3 security is how do you allow access to a user to assert in a user to a certain bucket which means this user is not having access to s3 at all but this user needs to be given access to a certain bucket how do we do it the same case applies to servers as well in few cases there could be an instance where a person is new to the team and you actually don't want them to access the production service now he's in the production group and by default he or she is granted access to that server but you specifically want to deny access to that production server till the time he or she is matured enough to access or understand the process understand the do's and don'ts before they can put their hands on the production server so how do we go about doing it so first we would categorize our instances well these are critical instances these are normal instances and we would actually uh put a tag on them that's how we categorize right so you put a tag on them put a tag saying well they are highly critical they are medium critical and they are not critical at all still there in production stuff like that and then you would pick the users who wants to or who should be or should not be given access to a certain server and you would actually allow the user to access or not access servers based on a specific tag in other words you can use actually tags in in the previous step we put tags on the critical server right so you would define that this user is not going to use this tag all right this user is not allowed to use the resources with this stack so that's how you would make your step forward so you would allow or deny based on the tags that you have put so in this case he or she will not be allowed to servers which are tagged critical servers so that's how you allow deny access to them and the same goes for bucket as well or if an organization is excessively using s3 for their data storage because of the benefit that it provides the cost and the durability you might get asked this question which is organizations would replicate the data from one region to another region for additional data durability and for having data redundancy not only for that they would also do that for dr purposes for disaster recovery if the whole region is down you still have the data available somewhere else and you can pick and use it some organizations would store data in different regions for compliance reasons to provide low latency access to their users who are local to that region stuff like that so when companies do replication how do you make sure that there is consistency in the replication how do you make sure that the replication is not failing and the data gets transferred for sure and there are logs for that replication this is something that the companies would use where they're excessively using s3 and they're fully relying on the replication in running their business and the way we could do it is we can set up a replication monitor it's actually a set of tools that we could use together to make sure that the cloud replication a region level replication is happening properly so this is how it happens now on this side on the left hand side we have the region 1 and on the right hand side we have region 2 and region one is the source bucket and region two is the destination bucket right so object is put in the source bucket and it has to go directly to the region to bucket or or made a copy in the region to bucket and the problem is sometimes it fails and there is no consistency between them so the way you would do it is connect these services together and create and cross replication or cross region replication monitor that actually monitors that actually monitors your environment so there are cloud watch that make sure that the data is uh moved no data is failing again there's cloud watch on the other end make sure that the data is moving and then we have the logs generated through cloud trail and that's actually written in dynamodb and if there is an error if something is failing you get notified through an sms or you get notified through an email using the sns service so that's how we could leverage these tools and set up and cross region replication monitor that actually monitors your data replication some common issues that company companies face in vpc is that we all know that i can use route 53 to resolve an ip address externally from the internet but by default the servers won't connect to the other servers using our custom dns name that does not do that by default so it's actually a problem there are some additional things that as an administrator or as an architect or as a person who uses it you will have to do and that's what we're going to discuss so the question could be a vpc is not resolving the server through the dns you can access it through the ip but not through the dns name and what could be the issue and how do you go about fixing it and you will be able to answer this question only if you have done it already it's a quick and simple step by default vpc does not allow that's the default feature and we will have to enable the uh dns hostname resolution before now this is for the custom dns not for the default dns that comes along this is for the custom dns so we will have to enable the dns host name resolution so our will have to enable dns hostname resolution so they actually resolve let's say i want to connect to a server1.simplylearn.com by default it's not allowed but if i enable this option then i will be able to connect to server one simplylearn.com if a company has vpcs in different regions and they have a head office in a central place and the rest of them are branch offices and they are connecting to the head office for access or you know for saving data or for accessing certain files or certain data or storing data all right so they would actually mimic the hubban spoke topology where you have the vpc which is centrally in an accessible region a centrally accessible region and then you would have a local vpcs or branch offices in different other regions and they get connected to the vpc in the central location and the question is how do you actually connect the multiple sites to a vpc and make communication happen between them by default it does not do that we know that vpcs they need to be paired between them in order to access the resources let's look at this picture right so i have like a customer network or branch offices in different parts and they get connected to a vpc that's fine so what we have achieved is those different offices the remote offices they are connecting to the vpc and they're talking but they can't connect or they can't talk to each other that's what we have built but the requirement is the traffic needs to or they should be able to talk to each other but they should not have direct connection between them which means that they will have to come and hit the vpc and then reach the other customer network which is in los angeles or which is in new york all right that's the requirement so that's possible with some architecting in the cloud so that's using vpn cloud hub you look at this dotted lines which actually allows customers or which actually allows the corporate networks to talk to each other through the vpc again by default it doesn't happen cloud hub is an architecture that we should be using to make this happen and what's the advantage of it as a central office or as the headquarters office which is in the vpc or headquarters data center which is in the vpc you have control or the vpc has control on who talks to who and what traffic can talk to i mean what traffic can be routed to the other head office stuff like that that centralized control is on the vpc the other question you could get asked is neiman explain some security products and features available in vpc well vpc itself is an security service it provides security service to the application but how do you actually secure the vpc itself that's the question and yes there are products that can actually secure the vpc or the vpc delivers those products to secure the application access to the vpc is restricted through a network access control list all right so that's and security product in vpc and a vpc has security groups that protects the instances from unwanted inbound and outbound traffic and network access control list protects the subnets from unwanted inbound and outbound access and there are flow logs we can capture in vpc that captures incoming and outgoing traffic through a vpc which will be used for later analysis as in what's the traffic pattern what's the behavior of the traffic pattern and stuff like that so there are some security products and features available in vpc now how do you monitor vpc vpc is a very important concept very important service as well everything sits in a vpc most of the service sits in a vpc except for lambda and s3 and dynamodb and couple of other services most of them sit in a vpc for security reason so how do you monitor your vpc how do you gain some visibility on your vpc well we can gain visibility on our vpc using vpc flow log that's the basic service as you see it actually captures what's allowed what's not allowed stuff like that which ips allowed which ip is not allowed stuff like that so we can gather it and we can use that for analysis and the other one is uh cloud watch and cloud watch logs the data transfers that happen so this is you know who gets allowed and who does not get allowed i mean the flow logs is who is allowed and who's not allowed that kind of detail and cloud watch gives information about the data transfer how much data is getting transferred we can actually pick unusual data transfers if there is a sudden hike in the graph there's a sudden hike and something happens at 12 on a regular basis and you weren't expecting it there's something suspicious it could be valid backups it could be a malicious activity as well so that's how you know by looking at cloudwatch logs and cloudwatch dashboard now let's talk about multiple choice questions when going for an interview you might sometimes find yourself that the company is conducting an online test based on the score they can put you to a panelist and then they would take it forward so we thought it will also include multiple choice questions to help you better handle such situation if you come across all right when you find yourself in such situation the key to clear them is to understand the question properly read between the lines that's what they say you know there can be like a big paragraph with three lines or ten lines you really got to understand what the question is about and then try to find answer for that question so that's a thumb rule number one and then the second rule is try to compare and contrast the services mentioned or try to compare and contrast the answers you can easily read out one or two answers and then you will be left with only two answers to decide from you know so that also helps you with time and that's all that also helps you with some precision in your answer so number one read between the lines number two compare and contrast the services and you'll be able to easily weed out the wrong ones so let's try answering this question suppose you are a game designer and you want to develop a game with a single digit millisecond latency which of the following database services would you choose so we know that the following are database services are good enough all right and it talks about millisecond latency that's a key point and the third thing is it's a game it could be a mobile game it's a game that you are trying to design and you need a millisecond latency and it has to be a database all right so let's talk about the options available rds rds is a database for sure is it good for a game design we'll come back to that neptune neptune is a graph a database service in amazon so that's kind of out of the equation and snowball is actually a storage all right it's it's a transport medium i would say so that's again out of the equation so the tie is between rds and dynamodb if we need to talk about rds rds is an a platform as a service it provides cost efficient resizable capacity but it's an sql database meaning the tables are kind of strict you know it's good for banking and other type of applications but not really good for anything that has to do with gaming so the only option left is dynamodb again it's the right answer dynamodb is actually an fast and flexible nosql database service and it provides a single digit millisecond latency at any scale and it's a database at the same time it's a key value store model database so the right answer is dynamodb all right let's look at the next question if you need to perform real-time monitoring of aws services and get actionable insights which service would you use all right let's list the services so it talks about real-time monitoring a firewall manager what does it provide now firewall manager is not really a monitor just like the name says it's a manager it manages multiple firewalls and aws guard duty is an threat detection service it does monitoring it does continuously monitor our environment but it monitors for threats all right only threats now let's talk about cloud watch a cloud watch is a service that helps to track metrics it's a service that is used to monitor the environment and give us a system wide visibility and also it helps us to store logs so at the moment it kind of looks like that could be the right answer we don't know that yet but i mean we have one more option left that's uh ebs so what's ebs ebs is a block storage elastic block store if we abbreviate ebs it's elastic block store so all three of them are easily out of the question the first one is to manage second one is to find threats of course it does monitoring so there's i mean if there is one relation between cloud watch and god duty that's monitoring so easily we can actually find ourselves slipped towards picking god duty but know that god duty is only for gaining security inside but not about gaining aws service inside so cloudwatch is a service that helps us to get a system wide or an aws wide or an account wide it has number of metrics we can monitor and get a very good insight of how a service is performing be it cpu be it ram bit network utilization beat connection failures cloud watch is a service that helps us perform a real-time monitoring and get some actionable insights on the services all right let's talk about this 33rd question as a web developer you are developing an app especially for the mobile platform all right there is a mention that this is especially for the mobile platform so a lot of services gets filtered out mobile platform right which of the following lets you add user sign up sign in and access control to your web and mobile app quickly and easily all right so this is all about signing in to your mobile app so if we need to read between the lines that's how we can read sign up or sign in into a mobile platform all right so we have like four options here uh shield aws massey aws inspector amazon cognito so let's try to weed out services which are not relevant to it so what's aws shield aws shield is actually a service that provides a ddos mitigation or ddos protection denial of service protection it's a security feature let's talk about the second option aws maxi is again a security service that uses machine learning to automatically discover and classify the data it again talks about security and this security is all about encrypting or saving the data does not come close with signing up and mobile platform all right let's talk about the other one aws inspector now aws inspector has something to do with apps it definitely has something to do with apps so kind of looks like that's relevant as of now so it actually helps with the improving the security and compliance of the apps that we deploy in the cloud so kind of looks like it could be because it has to do with apps the last one cognito now cognito is a service that actually lets the administrator to have control access over web and mobile apps and it's a service that helps us to sign up and uh sign in to an mobile and web app so that very much looks like we found the answer so cognito is a service that helps web app and mobile app for sign up and signing in and also gives the administrator to have control over who has i mean access control over the web and the mobile app pretty much we found it so it's cognito cognito is a service that helps us to set up sign up sign in and have access control over the users who would be using our mobile and web app all right how about this question uh you are an ml engineer or a machine learning engineer who is on the lookout for a solution that will discover sensitive information that your enterprise stores in aws and then uses nlp to classify that data and provide business related insights which among the following services would you choose so we have a bunch of services that's going to help us achieve or one of it is going to help us achieve the above requirement so it's a service that deals with machine learning you're a machine learning engineer who's looking for a service that will help you to discover information at your enterprise store so we're talking about storage discover information in store and then classify uh the data depending on severity the sensitivity classify the data so which service is that so firewall manager just like the name says it's a manager and the aws iam if we abbreviate it it's a identity and access management so it's identity and access management nothing to do with identifying sensitive data and managing it so the first two is already out of the equation and then uh the aw is massy we already had a quick definition description for aws massey that it's actually a security service that uses machine learning it kind of looks like it could be it it's a security service that uses machine learning and it discovers and classifies the sensitive information not only that it does not stop there it goes beyond and protects the sensitive data aws massey kind of looks like but we still have one more option to look at which is cloud hms cloud hms is also a security service kind of looks like that could be the answer as well and it enables us to generate encryption keys and save the data so kind of fifty percent of it it's a security service it encrypts helps us protect the data but aws maxi is right on spot it's a machine learning service it helps us to classify the data and also to protect the data so the answer for this question would be aws messy so hope you kind of get it how this is going so first we apply the thumb rule identify the question that's being asked read between the lines and then try to find the service that meets your requirement then finding the service is by first weeding out the wrong ones recollect everything that you've learned about the service and see how well that matches with those hints that you have picked up and if that doesn't match weed that out then you'll end up with two just to to decide from at some point and then it becomes easy for you to decide click on the question submit it and then move on to the other question in your interview all right so how about this one you are a system administrator in your company which is running most of its infrastructure on aws you are required to track your users and keep a look on how your users are being authenticated all right so this is where the problem statement starts right you need to keep track of how your users are being authenticated and you wish to create and manage aws users and use permissions to allow and deny their access to the aws resources right you are to give them permission number one and then i mean if we put them in the right order first giving them permissions and then tracking their usage let's see which of the service will help us achieve it iam is a service that helps us to looking at the permissions we can actually predict whether the user or the group will have servers or not so that helps us to get a track of who is able to use who's not able to use certain servers and all that stuff so it kind of looks like but we have other three options left let's look at aws firewall manager just like the name says it's actually a firewall manager it helps us to manage multiple firewalls simple as that and shield is a service it's a service that's used to protect denial of service or distributed denial of service an api gateway is a service that makes it easy for developers to create publish maintain and monitor and secure api so i mean it's completely on the api side very less on user and how you authenticate your user we can get that by looking at the name itself right if you abbreviate it or if you if you try to find a definition for the name api gateway you would get it it has to do with api but if you aggregate aws iam its identity and access management pretty much meets the requirement for the problem statement about its aws identity and access management that's the right answer all right let's look at this one if you want to allocate various private and public ip address in order to make them communicate with the internet and other instances you will use this service which of the following is this service so it talks about using public and private ip address so this service uses ip address and then this service helps us to allow and deny connections to the internet and to the other instances so you get the question is it let's pick the service that helps us achieve it route 53 route 53 is actually a dns service right so it's not a service that's used to allow or deny no it does not do that vpc vpc uses public and private ip address yes so kind of looks like a vpc helps us to allow i mean the security in vpc the security group the network access control list in a vpc that routing table in a vpc that actually helps us to allow or deny a connection to a particular ip address or to a particular service within the vpc or outside of the vpc so as of now it kind of looks like it could be but let's look at the other services what if if we find a service that closely matches to the above requirement than the amazon vpc gateway api gateway we know that it's a managed service that makes it easy for developers to create publish maintain and monitor apis and secure api so that has completely do with api not with ip cloud cloudfront we know about cloudfront that it's a content delivery network and it provides global distribution of servers where our content can be cached it could be video or bulk media or anything else they can be cached locally so users can easily access them and download them easily right so that's cloud front now at this point after looking at all four it looks like vpc is the right answer and in fact vpc is the right answer vpc has public ip address vpc can help us with a private ip address vpc can be used to allow deny connection based on the security group access control list and routing table it has so that's right answer is vpc all right how about this one this platform as a service or platform as a db service provides us with the cost efficient and resizable capacity while automating time consuming administrative tasks so this question is very clear it's a db service we got to look for and it's a service that can provide automating some of the time consuming tasks it has to be resizable at the same time so let's talk about amazon rational database it's a database kind of matches the requirement we can resize it as and when needed all right looks like it's a fit as of now it actually automates some of the time consuming work looks like it's a fit as of now let's move on to elastic cache and then try to see if that matches the definition that we've figured out about elastic cache it's actually a caching service it's again an in-memory data store which helps in achieving high throughput and low latency in memory data stores so it's not a full-blown database and it does not come with any amazon provisioned automation in it for automating any of the administration tasks no it does not come up with anything like that yeah we can resize the capacity as and when needed but automation it's not there yet and moreover it's not a database so that's out of the equation vpc is not a recessible one you know once we have designed vpc it's fixed it can't be resized so that's out of the equation and amazon glacier glacier is a storage but not a database right so that's again of the equation so that tie is kind of between amazon rational database service and amazon elastic cache because they both aid the database service but elastic cache is not a full-blown database it actually helps database but it's not a full-blown database so it's amazon relational database so that's the one which is a platform as a service it's the one which can be resized it's the one which can be used to automate the time consuming administrative tasks all right let's talk about this one which of the following is a means for accessing human researchers or consultants to help solve a problem on a contractual or a temporary basis all right let's read the question again which of the following is a means for accessing human researchers or consultant to help solve problems on a contractual or a temporary basis it's like signing task or hiring aws experts for a temporary job so let's try to find that kind of service in the four services that are listed amazon elastic mapreduce mapreduce is actually an framework service that makes it easy and cost effective to analyze large amount of data but that has nothing to do with accessing human researchers all right let's talk about mechanical term it's a web service that provides a human workforce that's the definition for it uh for example automation is good but not everything can be automated for something to qualify for automation it has to be an repetitive task one time task can't be automated or the time and money that you would be spending in automation is not worth it instead you could have done it manually so that does not qualify for automation and anything that requires intelligence right anything that's a special case right automation can do repetitive task automation can do precise work but it has to be repetitive task the scenario you know it should have been there already only then that can be executed but if it's a new scenario and it requires uh appropriate addressing then it requires human thought so we could hire researchers and consultants who can help solve a problem using amazon mechanical turk the other two are already out of the equation now dev pay is actually a payment system through amazon and multi-factor authentication as it says it's an authentication system so the right answer is amazon mechanical turk all right this sounds interesting let's look at this one this service is used to make it easy to deploy manage and scale containerized applications using kubernetes on aws which of the following is this aws service so it's a service to deploy manage and scale containerized application so it deals with containers it also should have the ability to use kubernetes which is and container orchestration service all right the first one amazon elastic container service kind of looks like it's the one the name itself has the word and the relation we're looking for elastic container service so this container service is an highly scalable high performance container orchestration service let's look at the other one aws batch it's a service that enables id professionals to schedule and execute uh batch processing i mean the name itself says that that's meant for batch processing elastic bean stock that's another service that helps us to deploy manage and scale but it helps us with easy two instances not with containerized instances so that's again out of the equation would light scale be a good tie for elastic container service what's light sale now lightsail is a service it's called as a virtual private server without a vpc it's called as a virtual private server it comes with a predefined compute storage networking capacity it's actually a server not a container right so at this point that also becomes out of the equation so it's amazon elastic container service that's the one that helps us to easily deploy manage scale container services and it helps us orchestrate the containers using kubernetes all right how about this one all right this service lets us to run code without provisioning or managing servers so no servers run code select the correct service from the below option all right so no servers but we should be able to run code amazon easy to auto scaling easy to auto scaling ec2 is elastic compute cloud which is a server and auto scaling is a service that helps us to achieve scaling the server so that's the definition for it could be that's out of the equation aw is lambda now lambda is a service it's actually event driven serverless computing platform and lambda runs code in response to the event that it receives and it automatically manages the compute resource that's required for that code as long as we have uploaded a code that's correct and set up events correctly to map to that code it's going to run seamlessly so that's about lambda it kind of looks like it could be the answer because lambda runs code we don't have to manage servers it manages servers by itself but we can't conclude as of now we have other two servers to talk about aws batch all right batches service that enables id professionals to run batch job we know that and about inspector amazon inspector it's actually a service that helps us to increase and identify any security issues and align our application with compliance well that's not the requirement of the question the requirement and the question was run code without provisioning your server and without any more space for confusion aws lambda is a service or is the service that runs code without provisioning and managing services right the right one would be aws lambda i'm very excited that you're watching this video and i'm equally glad that we were able to provide you a second part in aws interview questions all right let's get started so in an environment where there is a lot of automation infrastructure automation you'll be posted with this question how can you add an existing instance to a new auto scaling group now this is when you are taking an instance away from the auto scaling group to troubleshoot to fix a problem you know to look at logs or if you have suspended the auto scaling you know you might need to re-add that instance to the auto scaling group only then it's going to take part in it right only then the auto scaling is going to count it has part of it it's not a straight uh procedure you know when you remove them you know it doesn't get automatically re-added and i've had worked with some clients so when their developers were managing their own environment they had problems adding the instance back to the auto scaling group you know irrespective of what they tried the instance was not getting added to the auto scaling group and whatever they fixed that they were provided or whatever fix that they have provided were not you know getting encountered in the auto scaling group so like i said it's not a straight you know a click button procedure there are ways we'll have to do it so how can you add an existing instance uh to the auto scaling group there are a few steps that we need to follow so the first one would be to under the ec2 instance console right under the uh instance under actions in specific you know there's an option called attach to auto scaling group right if you have multiple auto scaling groups in your account or in the region that you're working in then you're going to be posted with the different auto scaling groups that you have in your account let's say you have five auto scaling groups for five different application you know you're going to be posted with five different auto scaling groups and then you would select the auto scaling the appropriate auto scaling group and attach the instance to that particular auto scaling group while adding to the auto scaling group if you want to change the instance type and that's possible as well sometimes when you want to add the instance back to the auto scaling group there would be requirement that you change the instance type to a better one to a better family to the better instance type you could do that at that time and after that you are or you have completely added the instance back to the auto scaling group so it's actually an seven step up process adding an instance back to the auto scaling group in an environment where they're dealing with migrating the instance or migrating an application or migrating an instance migrating and vm into the cloud you know if the project that you're going to work with deals with a lot of migrations you could be posted this question what are the factors you will consider while migrating to amazon web services the first one is cost is it worth moving the instance to the cloud given the additional bills and whistles features available in the cloud is this application going to use all of them is moving into the cloud beneficial to the application in the first place you know beneficial to the users who will be using the application in the first place so that's a factor to think of so this actually includes you know cost of the infrastructure and the ability to match the demand and supply transparency is this application and high demand you know is it going to be a big loss if the application becomes unavailable for some time so there are a few things that needs to be considered before we move the application to the cloud and then if uh the application does the application needs to be provisioned immediately is that an urge is there an urge to provision the application immediately that's something that needs to be considered if the application requires to go online if the application needs to hit the market immediately then we would need to move it to the cloud because in on-premises procuring or buying an infrastructure buying the bandwidth buying the switchboard you know buying an instance you know buying their softwares buying the license related to it it's going to take time at least like two weeks or so before you can bring up an server and launch an application in it right so the application cannot wait you know waiting means uh you know workforce productivity loss is it so we would want to immediately launch instances and put application on top of it in those case if your application is of that type if there is urge in making the application go online as soon as possible then that's a candidate for moving to the cloud and if the application or if the the software or if the product that you're launching it requires hardware it requires an updated hardware all the time that's not going to be possible in on premises we try to deal with legacy infrastructure all the time in on premises but in the cloud they're constantly upgrading their hardwares only then they can keep themselves up going in the market so they constantly the cloud providers are constantly updating their hardwares and if you want to be benefited of your application wants to be benefited by the constant upgrading of the hardwares making sure the hardware is as latest as possible the software version the licensing is as latest as possible then that's a candidate to be moved to the cloud and if the application does not want to go through any risk if the application is very sensitive to failures if the application is very much stacked to the revenue of the company and you don't want to take a chance in you know seeing the application fail and you know seeing the revenue drop then that's a candidate for moving to the cloud and business uh agility you know moving to the cloud at least half of the responsibility is now taken care by the provider in this case it's amazon at least half of the responsibility is taken care by them like if the hardware fails amazon makes sure that they're fixing the hardware immediately and notifications you know if something happens you know there are immediate notifications available that we can set it up and make ourselves aware that something has broken and we can immediately jump in and fix it so you see there are the responsibility is now being shared between amazon and us so if you want to get that benefit for your application for your organization for the product that you're launching then it needs to be moved to the cloud so you can get that benefit from the cloud the other question you could get asked is what is rto and rpo in aws they are essentially disaster recovery terms when you're planning for disaster recovery you cannot avoid planning disaster recovery without talking about rto and rpo now what's the rto what's the rpo in your environment or how do you define rto how do you define rpo or some general questions that get asked rto is recovery time objective rto stands for the maximum time the company is willing to wait for the recovery to happen or for the recovery to finish when and disaster strikes so rto is uh in the future right how much time is it going to take to fix and bring everything to normal so that's rtu on the other hand rpo is recovery point objective which is the maximum amount of data loss your company is willing to accept as measured in time rpo always refers to the backups the number of backups the the frequency of the backups right because when an outage happens you can always go back to the latest backup right and if the latest backup was before 12 hours you have lost the in between 12 hours of data data storage right so rpo is the acceptable amount if the company wants less rpo rpo is 1r then you should be planning on taking backups every one hour if rpo is 12 hours then you should be planning on taking backups every 12 hours so that's how rpo and rto you know helps disaster recovery the fourth question you could get asked is if you would like to transfer huge amount of data which is the best option among a snowball snowball edge and snowmobile again this is a question that get asked if the company is dealing with a lot of data transfer into the cloud or if the company is dealing with the migrating data into the cloud i'm talking about a huge amount of data data in petabytes snowball and all of the snowball series deals with the petabyte-sized data migrations so there are three options available as of now aws snowball is an data transport solution for moving high volume of data into and out of a specified aws region on the other hand aws snowball edge adds additional computing functions snowball is simple storage and movement of data and snowball edge has a compute function attached to it snow mobile on the other hand is an exabyte scale migration service that allows us to transfer data up to 100 petabytes that's like 100 000 terabytes so depending on the size of data that we want to transfer from our data center to the cloud we can hire we can rent any of these three services let's talk about some cloud formation questions this is a classic question how is aws cloud formation different from aws elastic bean stock you know from the surface they both look like the same you know you don't go through the console provisioning resources you don't you know you don't go through cli and provision resources both of them provision resources through click button right but underneath they are actually different services they support they aid different services so knowing that is going to help you understand this question a lot better let's talk about the difference between them and this is what you will be explaining to the interviewer or the panelist so the cloud formation the cloud formation service helps you describe and provision all the infrastructure resources in the cloud environment on the other hand elastic bean stock provides an simple environment to which we can deploy and run application cloud formation gives us an infrastructure and elastic bean stock gives us an small contained environment in which we can run our application and cloud formation supports the infrastructure needs of many different types of application like the enterprise application the legacy applications and any new modern application that you want to have in the cloud on the other hand the elastic bean stock it's a combination of developer tools they are tools that helps manage the life cycle of a single application so cloud formation in short is managing the infrastructure as a whole and elastic bean stock in short is managing and running an application in the cloud and if the company that you're getting hired is using uh cloud formation to manage their infrastructure using or if they're using infrastructure or any of the infrastructure as a code services then you would definitely face this question what are the elements of an aws cloud formation template so it has four or five basic uh elements right and the template is in the form of json or in yaml format all right so it has parameters it has outputs it has data it has resources and then the format or the format version or the file format version for the cloud formation template so parameter is nothing but it actually lets you to specify the type of ec2 instance that you want the type of rds that you want all right so ec2 is an umbrella rds is an umbrella and parameters within that ec2 and parameters within that rds are the specific details of the ec2 or the specific details of the rds service so that's what parameters in a cloud formation template and then the element of the cloud formation template is outputs for example if you want to output the name of an s3 bucket that was created if you want to output the name of the ec2 instance if you want to output the name of some resources that have been created instead of looking into the template instead of you know navigating through in the console and finding the name of the resource we can actually have them outputted in the result section so we can simply go and look at all the resources created through the template in the output section and that's what output values or output does in the cloud formation template and then we have resources resources are nothing but what defines what are the cloud components or cloud resources that will be created through this cloud formation template now ec2 is a resource rds is a resource and s3 bucket is a resource elastic load balancer is a resource and nat gateway is a resource vpc is a reserve so you see all these components are the resources and the resource section in the cloud formation defines what are the aws cloud resources that will be created through this cloud formation template and then we have a version a version actually identifies the capabilities of the template you know we just need to make sure that it is of the latest version type and the latest version is 0909 2010 that's the latest version number you'll be able to find that on the top of the cloud formation template and that version number defines the capabilities of the cloud formation template so just need to make sure that it's the latest all the time still talking about cloud formation this is another classic question what happens when one of the resource in a stack cannot be created successfully well if the resource in a stack cannot be created the cloud formation automatically rolls back and terminates all the resources that was created using the cloud formation template so whatever resources that were created through the cloud formation template from the beginning let's say we have created like 10 resources and the 11th resource is now failing cloud formation will roll back and delete all the 10 resources that were created previously and this is very useful uh when the cloud formation cannot you know go forward cloud formation cannot create additional resources because we have reached the elastic ip limits elastic ip limit per region is five right and if you have already used five ips and a cloud formation is trying to buy three more ips you know we've hit the soft limit till we fix that with amazon cloudformation will not be able to you know launch additional you know resources and additional ips so it's going to cancel and roll back everything that's true with a missing ec2 ami as well if an ami is included in the template and but the ami is not actually present then cloud formation is going to search for the mi and because it's not present it's going to roll back and delete all the resources that it created so that's what cloud formation does it simply rolls back all the resources that it created i mean if it sees a failure it would simply roll back all the resources that it created and this feature actually simplifies the system administration and layered solutions built on top of aws cloud formation so at any point we know that there are no orphan resources in the in in our environment you know because something did not work or because there was an you know cloud formation executed some there are no orphan resources in our account at any point we can be sure that if cloud formation is launching a resource and if it's going to fail and it's going to come back and delete all the resources it's created so there are no orphan resources in our account now let's talk about some questions in elastic block store again if the environment deals with a lot of automation you could be thrown this question how can you automate easy to backup using ebs it's actually a six step process to automate the ec2 backups we'll need to write a script to automate the below steps using aws api and these are the steps that should be found in the scripts first to get the list of instances and then and then the script that we are writing should be able to connect to aws using the api and list the amazon abs volumes that are attached locally to the instance and then it needs to list the snapshots of each volume make sure the snapshots are present and it needs to assign a retention period for the snapshot because over time the snapshots are going to be invalid right once you have some 10 latest snapshots any snapshot that you have taken before that 10 becomes invalid because you have captured the latest and 10 snapshot coverage is enough for you and then the fifth point is to create a snapshot of each volume create a new snapshot of each volume and then delete the old snapshot anytime a new snapshot gets created the oldest snapshot of the list needs to go away so we need to include options we need to include scripts in our script lines in our script that make sure that it's deleting the older snapshots which are older than the retention period that we are mentioning another question that you could see in the interview beat written interview beat online interview or beat and telephonic or face-to-face interview is what's the difference between ebs and instant store let's talk about ebs first ebs is kind of permanent storage the data in it can be restored at a later point when we save data in ebs the data lives even after the lifetime of the ec2 instance for example we can stop the instance and the data is still going to be present in ebs we can move the ebs from one instance to another instance and the data is simply going to be present there so abs is kind of permanent storage when compared to instance on the other hand instance store is a temporary storage and that storage is actually physically attached to the host of the machine ebs is an external storage an instant store is locally attached to the instance or locally attached to the host of the machine we cannot detach an instant store from one instance and attach it to another but we can do that with eba so that's a big difference one is permanent data and another one is ebs is permanent instant store is a volatile data and instant store with instant store we won't be able to detach the storage and attach it to another instance and another feature of instant store is data in an instant store is lost if the disk fails or the instance is stopped or terminated so instant store is only good for storing cache data if you want to store permanent data then we should think of using ebs and not instant store while talking about storage on the same lines this is another classic question how can you take backups of efs like ebs and if you can take backup how do you take that backup the answer is yes we can take efs to efs backup solution efs does not support snapshot like ebs efs does not support snapshot snapshot is not an option for efs elastic file system right we can only take backup from one efs to another efs and this backup solution is to recover from unintended changes or deletions of the efs and this can be automated but any data that we store in efs can be automatically replicated to another efs and once this efs goes down or gets deleted or data gets deleted or you know the whole ef is is for some reason interrupted or deleted we can recover the data from we can use the other efs and bring the application to consistency and to achieve this it's not an one-step configuration it's a cycle there are series of steps that's involved before we can achieve efs to efs backup the first thing is to sign in to the aws management console and under efs or click on efs to efs restore button from the services list and from there we can use the region selector in the console navigation bar to select the actual region in which we want to work on and from there ensure that we have selected the right template you know some of the templates would be you know efs to efs backup granular backups incremental backups right so there are some templates the kind of backups that you want to take do you want to take grant alert do you want to take increment backups stuff like that and then create a name to that solution the kind of backup that we have created and finally review all the configurations that you have done and click on save and from that point onwards the data is going to be copied and from that point onwards any additional data that you put is going to copy it and replicate it now you have an efs to efs backup this is another classic question in companies which deals with a data management there are easy options to create snapshots but deleting snapshots is not always an you know click button or an single step configuration so you might be facing a question like how do you auto delete old snapshots and the procedure is like this now as best practice we will take snapshots of ebs volume to s3 all snapshots get stored in s3 we know that now and we can use aws ops automator to automatically handle all snapshots the ops automator service it allows us to create copy delete ebs snapshots so there are cloud formation templates available for aws ops automator and this automator template will scan the environment and it would take snapshots it would you know copy the snapshot from one region to another region if you want i know if you're setting up a dr environment and not only that based on the retention period that we create it's going to delete the snapshots which are older than the retention period so life or managing snapshot is made a lot easier because of this ops automator cloud formation template moving into questions and elastic load balancer this again could be an uh question in the interview what are the different types of load balancers in aws and what's their use case what's the difference between them and as of now as we speak there are three types of load balancers which are available in aws the first one being application load balancer just like the name says the application load balancer works on the application layer and deals with the http and https request and it also supports part based routing for example simplylearn.com slash some webpage simplylearn.com another website so it's going to direct the path based on the slash value that you give in the urls that's path based routing so it supports that and not only that it can support a port-based colon 8080 colon 8081 or colon 8090 you know based on that port also it can take a routing decision and that's what application load balancer does on the other hand we have network load balancer and the network load balancer makes routing decisions at the transport level it's faster because it has very less thing to work on it works on lower osi layer it works on a lower layer so it has very less information to work with than compared with application layer so comparatively it's a lot faster and it handles millions of requests per second and after the load balancer receives the connection it selects a target group for the default rule using the flow hash routing algorithm it does simple routing right it does not do path-based or port-based routing it does simple routing and because of it it's faster and then we have classic load balancer which is kind of expiring as we speak amazon is discouraging people using classic load balancer but there are companies which are still using classic load balancer they are the ones who are the first one to step in to amazon when classic load balancer was the first load balancer or the only load balancer available at that point so it supports http https tcp ssl protocol and it has a fixed relationship between a load balance report and the container port so initially we only have classic load balancer and then at after some point amazon said instead of having one load balancer address all type of traffic we're going to have two load balances called as the child from the classic two load balancer and one is going to specifically address the application requirement and one is going to specifically address the network requirement and let's call it as application load balancer and network load balancer so that's how now we have two different load balancers talking about load balancer another classic question could be what are the different uses of the various load balancer in aws elastic load balancing there are three types of load balancer we just spoke about it application load balancer is used if we need a flexible application management and tls termination and network load balancer if we require extreme performance and the load balancing should happen on based on static ips for the application and classic load balancer is an old load balancer which is for people who are still running their environment from ec2 classic network now this is an older version of vpc or this is what was present before vpc was created easy to classic network is what was present before ec2 was created so they are the three types and they are the use cases of it let's talk about some of the security related questions you would face in the interview when talking about security and firewall in aws we cannot avoid discussion talking about waff web application firewall and you would definitely see yourself in this situation where you've been asked how can you use aws off in monitoring your aws applications waff or web application firewall protects our web application from common web exploits and vaf helps us control which traffic source your application should be allowed or a block which traffic from a certain source you know which source or which traffic from a certain source should be allowed or blocked your application with waff we can also create custom rules that blocks common attack patterns you know if it is a banking application it has a certain type of attacks and if it is simple data management data storage application it has uh i mean content management application it has a separate type of attack so based on the application type we can identify a pattern and create rules that would actually block that attack based on the rule that we create and waff can be used for three cases you know the first one is allow all requests and then block all request and count all requests for a new policy so it's also an monitoring and management service which actually counts all the policies or counts all the requests that matches a particular policy that we create and some of the characteristics we can mention in awsof are the origin ips and the strings that appear in the request we can allow block based on origin ip allow block based on strings that appear in the request we can allow block or count based on the origin country length of the request yeah we can block and count the presence of malicious scripts in an connection you know we can count the request headers or we can allow block a certain request header and we can count the presence of a malicious sql code in a connection that we get and that want to reach our application still talking about security what are the different aws im categories we can control using aws iam we can do the following one is create and manage im users and once the user database gets bigger and bigger we can create and manage them in groups and in im we can use it to manage the security credentials kind of setting the complexity of the password you know setting additional authentications you know like mfa and you know rotating the passwords no resetting the password there are a few things we could do with iam and finally we can create policies that actually grants access to aws services and resources another question you will see is what are the policies that you can set for your user's password so some of the policies that we can set for the user password is at the minimum length or you know the complexity of the password by at least having one number or one special characters in the password so that's one and then the requirement of a specific character types including you know uppercase lowercase number and non-alphabetic characters so it becomes very hard for somebody else to guess what the password uh would be and and try to hack them so we can set the length of the password we can set the complexity in the password and then we can set an automatic expiration of the password so after a certain time the user is forced to create a new password so the password is not still old and easy to guess in the environment and we can also set settings like the user should contact the admin i mean when the password is about to expire so you know you can get a hold of how the user is setting their password is it having good complexity in it is it meeting company standards or there are few things that we can control and set for the users when the users are setting or recreating the password another question that could be posted in an interview so to understand your understanding of iem is what's the difference between an iem role and an im user let's talk about iem user let's start small and then go big or let's start simple and then talk about the complex one the im user has a permanent long term credential and it's used to interact directly with aws services and on the other hand iem role is an iem entity that defines a set of permissions for making aws service request so i am user is an permanent credential and role are temporary credentials and iem user has full access to all aws iem functionalities and with role trusted entities such as iem users application or aws services assume the role so when an im user is given an permission you know it sticks within the iem user but with roles we can give permissions to applications we can give permissions to users in the same account in a different account corporate id we can give permissions to ec2 s3 rds vpc and lot more role is wide and im user is is not so wide you know it's very constrained only for that i am user let's talk about managed policies in aws managed policies there are two types you know customer managed and amazon managed so manage policies are i am resources that express permissions using the iam policy language and we can create policies edit them manage them manage them separately from the im user group and roles which they are attached to so they are something that we can do to managed policies if it is customer managed and we can now update policy in one place and the permissions automatically extend to all the attached entries so i can have like three services four services point to a particular policy and if i edit that particular policy it's gonna reflect on those three or four services so anything that i allow is going to be allowed for those four services anything that i denied is going to be denied for uh the four services imagine what would be without the i am managed policy will have to go and specifically allow deny on those different instances four or five times depending on the number of instances that we have so like i said there are two types of managed policies one is managed by us which is customer managed policies and then the other is managed by aws which is aws managed policy this question can you give an example of an iem policy and a policy summary this is actually to test how well-versed are you with the aws console the answer to that question is look at the following policy this policy is used to grant access to add update and delete objects from a specific folder now in this case name of the folder is example folder and it's present in a bucket called example bucket so this is an iam policy on the other hand the policy summary is a list of access level resource and conditions for each service defined in a policy so im policy is all about one particular resource and the policy summary is all about multiple resources with iem policy it was only talking about s3 bucket and one particular s3 bucket here it talks about cloud formation template cloud watch logs ec2 elastic bean stock services summary summary of resources and the permissions and policies attached to them that's what policy summary is all about another question could be like this what's the use case of iam and how does im help your business two important or primary work of im is to help us manage iam users and their access it provides a secure access to multiple users to their appropriate aws resources so that's one it does and the second thing it does is manage access for federated users federated users or non iam users and through iam we can actually allow and provide secured access to resources in our aws account to our employees without the im user no they could be authenticated using the active directory they could be authenticated using the facebook credential google credential amazon credential and a couple of other credentials third party identity management right so we could actually trust them and we could give them access to our account based on the trust relationship that we have built with the other identity of systems right so two things one is manage users and their access for manage iam user and their access in our aws environment and second is manage access for federated users who are non-iam users and more importantly im is a free service and with that will only be charged for the use of the resources not for the im username and password that we create all right let's now talk about some of the questions in route 53 one classic question that could be asked in an interview is what is the difference between latency based routing and geodns or jio based dns routing now the geo based dns routing takes routing decisions on the basis of the geographic location of the request and on the other hand the latency based routing utilizes latency measurements between networks and data centers now latency based routing is used where you want to give your customers the lowest latency as possible so that's when we would use latency based routing and on the other hand the geo-based routing is when we want to direct customers to different websites based on the country they are browsing from you know you could have you know two different or three different websites for the same url you know take amazon the shopping website for example when we go to amazon.com from in the us it directs us to the us web page where the products are different the currency is different right and the flag and a couple of other advertisements that shows up are different and when we go to amazon.com from india it gets directed to the amazon.com indian site where again the currency the product and the advertisements they're all different right so depending on the country they're trying to browse if you want to direct customers to two or three different websites we would use a geo-based routing another use case of jio based routing is if you have a compliance that you should handle all the dns requires sorry if you should handle all the requests you know from a country within the country then you would do jio based routing now you wouldn't direct the customer to a server which is in another country all right you would direct the customer to a server which is very local to them that's another use case of jio based routing and like i said for latency based routing the whole goal or aim is to achieve minimum end user latency if you are hired for the architect role and if that requires working lot on the dns then you could be posted with this question what is the difference between domain and a hosted zone a domain is actually a collection of data describing a self-contained administrative and technical unit on the internet right so for example you know simplylearn.com is actually a domain on the other hand hosted zone is actually an container that holds information about how you want to route traffic on the internet to a specific domain for example lms.simplylearn.com is an hosted zone whereas simplylearn.com is a domain so in other words in hosted zone you would see the domain name plus and a prefix to it lms is a prefix here ftp is a prefix mail.simplylearn.com as a prefix so that's how you would see a prefix in hosted zones another question from another classic question from route 53 would be how does amazon drop 53 provide high availability and low latency the way amazon rav 53 provides high availability and low latency is by globally distributed dns servers amazon is a global service and they have dna services globally any customer doing a query from different parts of the world they get to reach an dns server which is very local to them and that's how it provides low latency now this is not true with all the dns providers there are dns providers who are very local to a country who are very local to a continent so they don't they generally don't provide low latency service right it's always high latency it's low latency for local users but anybody browsing from a different country or a different continent it's going to be high latency for them but that's not again true with amazon amazon is a globally distributed dns provider it has dns servers global wide and like i said it has optimal locations it has got global servers or in other words it has got servers around the globe different parts in the globe and that's how they are able to provide high availability and because it's not running on just one server but on many servers they provide high availability and low latency if the environment that you're going to work on is going to take a lot of configuration backups environmental backups then you can expect questions in aws config a classic question would be how does aws config work along with aws cloudtrail aws cloudtrail actually records user api activity on the account and you know any http https access or any any sort of access you know that's made to the cloud environment that's recorded in the cloud trail in other words any api calls the time is recorded the type of call is recorded and what was the response given was it a failure was it successful they also get recorded in cloudtrail it's actually a log it actually records the activity in your cloud environment on the other hand config is an uh point in time configuration details of your resources for example at a given point what are all the resources that were present in my environment what are all the resources or what are the configuration in those resources at a given point they get captured in aws conflict right so with that information you can always answer the question what did my aws resource look like at a given point in time that question gets answered when we use aws config on the other hand with cloudtrail uh you can answer the question i mean by looking at the cloudtrail or with the help of cloudtrail you can easily answer the question who made an apa call to modify this resource that's answered by cloud trail and with the cloud trail we can detect if a security group was incorrectly configured and who did that configuration let's say there happened to be a downtime and you want to identify let's say there happened to be a downtime and you want to identify who made that change in the environment you can simply look at cloudtrail and find out who made the change and if you want to look at how the environment looks like before the change you can always look at aws config can aws configure or aws config aggregate data across different aws accounts yes it can now this question is actually to test whether you have used aws config or not i know some of the services are very local is it some of these services are availability zone specific some of them are regional specific and some of them are global services in amazon and though some of the services are region services you still can do some changes you know add some configuration to it and collect regional data in it for example s3 is a regional service but still you can collect logs from all other regions into an s3 bucket in one particular region that's possible and cloud trail is and cloud watch is an regional service but still you can with some changes to it with some adding permissions to it you can always monitor the cloud watch that belongs to cloud watch logs that belongs to other regions you know they're not global by default but you can do some changes and make it global similarly aws config is a service that's a region based service but still you can make it act globally you can aggregate data across a different region and different accounts in an aws config and deliver the updates from different accounts to one s3 bucket and can access it from there aws config also works or integrates seamlessly with sns topic so you know anytime there is a change anytime a new data gets collected you can always notify yourself or notify a group of people about the new log or the new config or new edit that happened in the environment let's look at some of the database questions you know database should be running on reserved instances so whether you know that fact or not the interviewer wants to understand how well you know that fact by asking this question how are reserved instances different from on-demand db instances reserved instances and on-demand instances are exactly the same when it comes to their function but they only differ based on how they are built reserved instances are purchased for one year or three year reservation and in return we get a very low per hour pricing because we're paying upfront it's generally said that reserved instance is 75 percentage cheaper than on-demand instance and amazon gives you that benefit because you know you're committing for one year and sometimes you're paying in advance for the whole year on the other hand on demand instances are built on an oddly early price talking about auto scaling how will you understand the different types of auto scaling the interviewer might ask this question which type of scaling would you recommend for rds and y the two types of scaling as you would know now are vertical and horizontal and in vertical scaling we can vertically scale up the master database with a couple of clicks all right so that's vertical scaling vertical scaling is keeping the same node and making it bigger and bigger if previously it was running on t2 micro now we would like to run it on m3 two times large instance previously it had one virtual cpu one gigabit now it's going to have eight virtual cpu and 30 gigabit of ram so that's vertical scaling on the other hand horizontal scaling is adding more nodes to it previously it was running on one vm now it's going to run on 2 3 10 vms all right that's horizontal scaling so database can only be scaled vertically and there are 18 different types of instances we can resize our rds to right so this is true for ids mysql postgres sql mariadb oracle microsoft sql servers there are 18 type of instances we can vertically scale up to on the other hand horizontal scaling are good for replicas so they are read-only replicas we're not going to touch the master database we're not going to touch the primary database but i can do horizontal scaling only with amazon aurora and i can add additional read replicas i can add up to 15 read replicas for amazon aurora and up to five read replicas for rds mysql postgres sql and marie db rds instances and when we add replica we are horizontally scaling adding more nodes right read only nodes so that's horizontal scaling so how do you really decide between vertical and horizontal scaling if you're looking in to increase the storage and the processing capacity we'll have to do a vertical scaling if you're looking at increasing the performance or of the read heavy database we need to be looking for horizontal scaling or we need to be implementing horizontal scaling in our environment still talking about database this is another good question you can expect in the interview what is the maintenance window in amazon rds will your db instance be available during the maintenance event all right so this is really to test how well you have understood the sla how well you have understood the amazon rdas uh the failover mechanism of amazon rdas uh stuff like that so audio's maintenance window it lets you decide when a db instance modification a database engine upgrades or software patching has to occur and you you actually get to decide should it happen at 12 in the night or should it happen at afternoon should it happen early in the morning should it happen in the evening you actually get to decide an automatic scheduling by amazon is done only for patches that are security and durability related sometimes amazon takes down and does automatic scheduling if you know if there is a need for a patch update that deals with security and durability and by default the maintenance window is is for 30 minutes and the important point is the db instance will be available during that event because you're going to have primary and secondary right so when that upgrade happens amazon would shift the connection to the secondary do the upgrade and then switch back to the primary another classic question would be what are the consistency models in dynamodb in dynamodb there is eventual consistency read this eventual consistency model it actually maximizes your read throughput and the best part with eventual consistency is all copies of data reach consistency within a second and sometimes when you write and when you're you know trying to read immediately chances that you you would still be reading the old data that's eventual consistency on the other hand there is another consistency model called the strong consistency or strongly consistent read where there is going to be a delay in writing the data you know making sure the data is written in all places but it guarantees one thing that is once you have done a write and then you're trying to do a read it's going to make sure that it's going to show you the updated data not the old data and you can be guaranteed of it that it is going to show the updated data and not the old data that's strongly consistent still talking about database talking about nosql dynamodb or nosql database which is dynamodb and amazon you could be asked this question what kind of query functionality does dynamodb support dynamodb supports get and put operation dynamodb supports or dynamodb provides flexible querying by letting you query on non-primary key attributes using global secondary index and local secondary indexes a primary key can be either a single attribute partition key or a composite partition sort key in other words a dynamodb indexes a composite partition sort key as a partition key element and a sort key element and by holding the partition key you know when doing a search or when doing a query by holding the partition k element constant we can search across the sort key element to retrieve the other items in that table and the composite partition sort key should be a combination of user id partition and a timestamp so that's what the composite partition sort key is made of let's look at some of the multiple choice questions you know sometimes some companies would have an a written test or an mcq type online test before they call you for at the first level or before they call you for the second level so these are some classical questions that companies asked or companies ask in their multiple choice online questions let's look at this question as a developer using this pay-per-use service you can send store and receive messages between software components which of the following is being referred here let's look at it right we have aws step functions amazon mq amazon simple queue service amazon simple notification service let's say the question again as a developer using this pay-per-use service so the service that we are looking for is a pay-per-view service you can send store and retrieve messages between two software components kind of like a queue there so what would be the right answer it would be amazon simple queue service now amazon simply queue service is the one that's used to decouple at the environment it breaks the tight coupling and then it introduces decoupling in that environment by providing a queue or by inserting a queue between two software components let's look at this other question if you would like to host a real-time audio and video conferencing application on aws it's an audio and video conferencing application on aws this service provides you with a secure and easy to use application what is this service let's look at the options they are amazon chime amazon workspace amazon mq amazon app stream but you might tend to look at amazon app stream because it's real time and a video conference but it's actually for a different purpose is actually amazon chime that lets you create chat and create a chat board and then collaborate with the security of the aws services so it lets you do the audio it lets you do the video conference all supported by aws security features it's actually amazon chime let's look at this question as your company's aw solution architect you are in charge of designing thousands of individual jobs which are similar which of the following service best serves your requirement in earlier ec2 autoscaling aws snowball aws fargate aws batch let's read the question again as your company's aw solution architect you are in charge of designing thousands of individual jobs which are similar it looks like it's batch service let's look at the other options as well aw snowball is actually an storage uh transport service ec2 auto scaling is you know in introducing scalability and elasticity in the environment and aws fargate is container services aws batch is the one is being referred here that actually runs thousands of individual jobs which are similar aws batch it's the right answer but let's look at the other one you are a machine learning engineer and you're looking for a service that helps you build and train machine learning models in aws which among the following are we referring to so we have amazon sagemaker and aws deep lens amazon comprehend aws device farm let's read the question again you are a machine learning engineer and you're looking for a service that helps you build and train machine learning models in aws which among the following are referred here the answer is sage maker it provides every developer and data scientist with the ability to build train and deploy mission learning models quickly that's what sagemaker does now for you to be familiar with you know the the products i would recommend you to you know simply go through the product description you know there's one page available on amazon that explains all the products a quick neat and simple that really helps you to be very familiar with you know what the product is all about and what it is capable of you know is it a db service is it a machine learning service or is it a monitoring service is it a developer service stuff like that so get that information get that details before you attend an interview and that should really help to answer or face such questions with a great confidence so the answer is amazon sagemaker because that's the one that provides developers and a data scientist the ability to build a train and deploy machine learning models quickly as possible all right let's look at this one let's say that you are working for your company's id team and you are designated to adjust the capacity of the aws resource based on the incoming application and network traffic how do you do it so what's the service that's actually helps us to adjust the capacity of the aws resource based on the incoming application let's look at it amazon vpc amazon iam amazon inspector amazon elastic load balancing amazon vpc is a networking service amazon iam is an username password authentication amazon inspector is a service that actually does security audit in our environment and amazon elastic load balancer is a service that helps in scalability that's in one way you know indirectly that helps in increasing the availability of the application right and monitoring it monitoring you know how much requests are coming in through the elastic load balancer we can actually adjust the environment that's running behind it so the answer is going to be amazon elastic load balancer all right let's look at this question this cross-platform video game development engine that supports pc xbox playstation ios and android platforms allows developers to build and host their games on amazon's servers so we have amazon game lift amazon green grass amazon lumber yard amazon sumerian let's read the question again this cross-platform video game development engine that supports pc xbox playstation ios and android platforms allows developers to build and host their games on amazon servers the answer is amazon lumberyard this lumberyard is an free aaa gaming engine deeply integrated with aws and twitch with full source this lumbar yard provides a growing set of tools that helps you create and highest game quality applications and they connect to a lot of games and vast compute and storage in the cloud so it's that service they are referring to let's look at this question you are the project manager of your company's cloud architect team you are required to visualize understand and manage your aws cost and usage over time which of the following service will be the best fit for this we have aws budgets we have aws cost explorer we have amazon work mail we have amazon connect and the answer is going to be cost explorer now cost explorer is an option in the amazon console that helps you to visualize and understand and even manage the aws cost over time who's spending more who's spending less and what is the trend what is the projected cost for the coming month all these can be visualized in aws cost explorer let's look at this question you are a chief cloud architect at your company and how can you automatically monitor and adjust computer resources to ensure maximum performance and efficiency of all scalable resources so we have a cloud formation as a service we have aws aurora as a solution we have aws auto scaling and amazon api gateway let's read the question again you're the chief cloud architect at your company how can you automatically monitor and adjust computer resources how can you automatically monitor and adjust computer resources to ensure maximum performance and efficiency of all scalable resources this is an easy question to answer the answer is auto scaling right that's a basic service and solution architect course is it auto scaling is the service that helps us to easily adjust monitor and ensure the maximum performance and efficiency of all scalable resources it does that by automatically scaling the environment to handle the inputs let's look at this question as a database administrator you will use a service that is used to set up and manage databases such as mysql maya db and postgres sql which service are we referring to amazon aurora amazon elastic cache aws rds aws database migration service amazon arora is amazon's flavor of the rds service and elastic cache is is the caching service provided by amazon they are not full-fledged database and database migration service just like the name says it helps to migrate the database from on premises to the cloud and from one database flavor to another database flavor amazon rds is the service is the console is the service is the umbrella service that helps us to set up manage databases like mysql money db and postgres sql it's amazon rts let's look at this last question a part of your marketing work requires you to push messages to onto google facebook windows and apple through apis or aws management console you will use the following service so the options are aws cloudtrail aws config amazon chime aws simple notification service it says a part of your marketing work requires you to push messages it's dealing with pushing messages to google facebook windows and apple through apis or aws management console you will use the following service it's simple notification service simple notification service is an message pushing a service and sqs is pulling similarly sns is pushing right here it talks about a pushing system that pushes messages to google facebook windows and apple through api and it's going to be a simple notification system or a simple notification service and with that we have come to the end of this video on aws full course i hope it was informative and interesting if you have any questions about the topics that were covered in this video please ask away in the comment section below our team will help you solve your queries thanks for watching stay safe and keep [Music] learning hi there if you like this video subscribe to the simply learn youtube channel and click here to watch similar videos turn it up and get certified click here

Channel: Simplilearn

Views: 67,591

Rating: 4.9360428 out of 5

Keywords: aws full course, aws tutorial for beginners, aws training for beginners, aws certified solutions architect, aws full course tutorial, aws full crash course, aws solution architect certification, aws interview questions, aws ec2 tutorial, aws ec2 tutorial for beginners, aws lambda, aws lambda tutorial, aws s3, aws s3 tutorial for beginners, aws iam, aws iam tutorial, aws cloudformation, aws cloud computing, aws simplilearn, simplilearn

Id: uQdzcIf_KII

Length: 656min 49sec (39409 seconds)

Published: Sun Dec 27 2020