AWS Full Course 2022 | AWS Tutorial For Beginners 2022 | AWS Training For Beginners | Simplilearn

hello everyone welcome to this interesting video on aws full course of 2022. in this video we will explore various important aspects revolving around aws and its related concepts in depth but before diving deep into its concepts we will first cover the basics and have an introduction to what is aws followed by exploring cloud computing services then we will explore some of the very important services provided by aws and various categories like compute storage database and networking following that we will explore and understand the complete mechanism and architecture of some of its very important features related to containers storage and security that are aws ec2 aws s3 aws im aws cloud formation aws route 53 aws ecs aws bean stock aws vpc aws sage maker aws cloud front aws auto scaling and aws redshift after which we will have a look at the comparison between aws and other cloud platforms like azure and gcp to understand the individual identities and how they differ from each other based on various parameters and we'll also see how kubernetes are implemented on aws and after that we will focus on how we can adopt aws in our career and pave our path towards becoming aws cloud practitioner or ews solution architect after that we will discuss the top 10 reasons why ews is a better option to go with we will then conclude this video by discussing the essential interview questions and answer to help every individual clear an interview with full confidence by the end of this video i can assure you that all your aws career related queries would have been answered for this training with me i have a experienced aws specialist sam and rahul together we'll walk you through the different crucial aws keynotes so let's start with an exciting video on aws full course of 2022 but before we begin make sure to subscribe to our youtube channel and hit the bell icon and never miss an update from simply learn meet rob he runs an online shopping portal the portal started with a modest number of users but has recently been seeing a surge in the number of visitors on black friday and other holidays the portal saw so many visitors that the servers were unable to handle the traffic and crashed is there a way to improve performance without having to invest in a new server wondered rob a way to upscale or downscale capacity depending on the number of users visiting the website at any given point well there is amazon web services one of the leaders in the cloud computing market before we see how aws can solve rob's problem let's have a look at how aws reach the position it is at now aws was first introduced in 2002 as a means to provide tools and services to developers to incorporate features of amazon.com to their website in 2006 its first cloud services offering was introduced in 2016 aws surpassed its 10 billion revenue target and now aws offers more than 100 cloud services that span a wide range of domains thanks to this the aws cloud service platform is now used by more than 45 of the global market now let's talk about what is aws aws or amazon web service is a secure cloud computing platform that provides computing power database networking content storage and much more the platform also works with a pay as you go pricing model which means you only pay for how much of the services offered by aws you use some of the other advantages of aws are security aws provides a secure and durable platform that offers end-to-end privacy and security experience you can benefit from the infrastructure management practices born from amazon's years of experience flexible it allows users to select the os language database and other services easy to use users can host applications quickly and securely scalable depending on user requirements applications can be scaled up or down aws provides a wide range of services across various domains what if rob wanted to create an application for his online portal aws provides compute services that can support the app development process from start to finish from developing deploying running to scaling the application up or down based on the requirements the popular services include ec2 aws lambda amazon light cell and elastic beanstalk for storing website data rob could use aws storage services that would enable him to store access govern and analyze data to ensure that costs are reduced agility is improved and innovation accelerated popular services within this domain include amazon s3 ebs s3 glacier and elastic file storage rob can also store the user data in a database with aw services which he can then optimize and manage popular services in this domain include amazon rds dynamodb and redshift if rob's businesses took off and he wanted to separate his cloud infrastructure or scale up his work requests and much more he would be able to do so with the networking services provided by aws some of the popular networking services include amazon vpc amazon route 53 and elastic load balancing other domains that aws provides services in are analytics blockchain containers machine learning internet of things and so on and there you go that's aws for you in a nutshell now before we're done let's have a look at a quiz which of these services are incorrectly matched one two three four we'll be pinning the question in the comment section comment below with your answer and stand a chance to win an amazon voucher several companies around the world have found great success with aws companies like netflix twitch linkedin facebook and bbc have taken advantage of the services offered by aws to improve their business efficiency and thanks to their widespread usage aws professionals are in high demand they're highly paid and earn up to more than one hundred and twenty seven thousand dollars per annum once you're aws certified you could be one of them too hello everyone let me introduce myself as sam a multi-platform cloud architect and trainer and i'm so glad and i'm equally excited to talk and walk you through the session about what aws is and talk to you about some services and offerings and about how companies get benefited by migrating their applications and infra into aws so what's aws let's talk about that now before that let's talk about how life was without any cloud provider and in this case how life was without aws so let's walk back and picture how things were back in 2000 which is not so long ago but a lot of changes a lot of changes for better had happened since that time now back in 2000 a request for a new server is not an happy thing at all because a lot of money a lot of validations a lot of planning are involved in getting a server online or up and running and even after we finally got the server it's not all said and done a lot of optimization that needs to be done on that server to make it worth it and get a good return on investment from that server and even after we have optimized for a good return on investment the work is still not done there will often be a frequent increase and decrease in the capacity and you know even news about our website getting popular and getting more hits it's still an bittersweet experience because now i need to add more servers to the environment which means that it's going to cost me even more but thanks to the present day cloud technology if the same situation were to happen today my new server it's almost ready and it's ready instantaneously and with the swift tools and technologies that amazon is providing uh in provisioning my server instantaneously and adding any type of workload on top of it and making my storage and server secure you know creating a durable storage where data that i store in the cloud never gets lost with all that features amazon has got our back so let's talk about what is aws there are a lot of definitions for it but i'm going to put together a simple and a precise definition as much as possible now let me iron that out cloud still runs on and hardware all right and there are certain features in that infrastructure in that cloud infrastructure that makes cloud cloud or that makes aws a cloud provider now we get all the services all the technologies all the features and all the benefits that we get in our local data center like you know security and compute capacity and databases and in fact you know we get even more cool features like uh content caching in various global locations around the planet but again out of all the features the best part is that i get or we get everything on a pay as we go model the less i use the less i pay and the more i use the less i pay per unit very attractive isn't it right and that's not all the applications that we provision in aws are very reliable because they run on a reliable infrastructure and it's very scalable because it runs on an on-demand infrastructure and it's very flexible because of the designs and because of the design options available for me in the cloud let's talk about how all this happened aws was launched in 2002 after the amazon we know as the online retail store wanted to sell their reminding or unused infrastructure as a service or as an offering for customers to buy and use it from them you know sell infrastructure as a service the idea sort of clicked and aws launched their first product first product in 2006 that's like four years after the idea launch and in 2012 they held a big sized customer even to gather inputs and concerns from customers and they were very dedicated in making those requests happen and that habit is still being followed it's still being followed as a reinvent by aws and at 2015 amazon announced its revenue to be 4.6 billion and in 2015 through 2016 aws launched products and services that helped migrate customer services into aws well there were products even before but this is when a lot of focus was given on developing migrating services and in the same year that's in 2016 amazon's revenue was 10 billion and not but not the least as we speak amazon has more than 100 products and services available for customers and get benefited from all right let's talk about the services that are available in amazon let's start with this product called s3 now s3 is a great tool for internet backup and it's it's the cheapest storage option in the object storage category and not only that the data that we put in s3 is retrievable from the internet s3 is really cool and we have other products like migration and data collection and data transfer products and here we can not only collect data seamlessly but also in a real-time way monitor the data or analyze the data that's being received that they're cool products like aws data transfers available that helps achieve that and then we have products like ec2 elastic compute cloud that's an recessable computer where we can anytime anytime after the size of the computer based on the need or based on the forecast then we have simple notification services systems and tools available in amazon to update us with notifications through email or through sms now anything anything can be sent through email or through sms if you use that service it could be alarms or it could be service notifications if you want stuff like that and then we have some security tools like kms key management system which uses aes 256 bit encryption to encrypt our data at rest then we have lambda a service for which we pay only for the time in seconds seconds it takes to execute our code and uh we're not paying for the infrastructure here it's just the seconds the program is going to take to execute the code for the short program we'll be paying in a milliseconds if it's a bit bigger program we'll be probably paying in 60 seconds or 120 seconds but that's not cheap lot simple and lots cost effective as against paying for service on an hourly basis which a lot of other services are well that's cheap but using lambda is a lot cheaper than that and then we have services like route 53 a dns service in the cloud and now i do not have to maintain and dns account somewhere else and my cloud environment with aws i can get both in the same place all right let me talk to you about how aws makes life easier or how companies got benefited by using aws as their i.t provider for their applications or for the infrastructure now uniliver is a company and they had a problem right and they had a problem and they picked aws as a solution to their problem right now this company was sort of spread across 190 countries and they were relying on a lot of digital marketing for promoting their products and their existing environment their legacy local environment proved not to support their changing id demands and they could not standardize their old environment now they chose to move part of their applications to aws because they were not getting what they wanted in their local environment and since then you know rollouts were easy provisioning new applications became easy and even provisioning infrastructure became easy and they were able to do all that in push button scaling and needless to talk about backups that are safe and backups that can be securely accessed from the cloud as needed now that company is growing along with aws because of their swift speed in rolling out deployments and being able to access secure backups from various places and generate reports and in fact useful reports out of it that helps their business now on the same lines let me also talk to you about kellogg's and how they got benefited by using amazon now kellogg's had a different problem it's one of its kind now their business model was very dependent on an infrared that will help to analyze data really fast right because they were running promotions based on the analyzed data that they get so they being able to respond to the analyzed data as soon as possible was critical or vital in their environment and luckily sap running on hana environment is what they needed and you know they picked that service in the cloud and that's sort of solved the problem now the company does not have to deal with uh maintaining their legacy infra and maintaining their heavy compute capacity and maintaining their database locally all that is now moved to the cloud or they are using cloud as their i.t service provider and and now they have a greater and powerful it environment that very much complements their business let me start the session with this scenario let's imagine how life would have been without spotify for those who are hearing about spotify for the first time a spotify is an online music service offering and it offers instant access to over 16 million licensed songs spotify now uses aws cloud to store the data and share it with their customers but prior to aws they had some issues imagine using spotify before aws let's talk about that back then users were often getting errors because spotify could not keep up with the increased demand for storage every new day and that led to users getting upset and users cancelling the subscription the problem spotify was facing at that time was their users were present globally and were accessing it from everywhere and they had different latency in their applications and spotify had a demanding situation where they need to frequently catalog the songs released yesterday today and in the future and this was changing every new day and the songs coming in rate was about 20 000 a day and back then they could not keep up with this requirement and needless to say they were badly looking for a way to solve this problem and that's when they got introduced to aws and it was a perfect fit and match for their problem aws offered at dynamically increasing storage and that's what they needed aws also offered tools and techniques like storage life cycle management and trusted advisor to properly utilize the resource so we always get the best out of the resource used aws addressed their concerns about easily being able to scale yes you can scale the aws environment very easily how easily one might ask it's just a few button clicks and aws solved spotify's problem let's talk about how it can help you with your organization's problem let's talk about what is aws first and then let's bleed into how aws became so successful and the different types of services that aws provides and what's the future of cloud and aws in specific let's talk about that and finally we'll talk about a use case where you will see how easy it is to create a web application with aws all right let's talk about what is aws aws or amazon web services is a secure cloud service platform it is also pay as you go type billing model where there is no upfront or capital cost we'll talk about how soon the service will be available well the service will be available in a matter of seconds with aws you can also do identity and access management that is authenticating and authorizing a user or a program on the fly and almost all the services are available on demand and most of them are available instantaneously and as we speak amazon offers 100 plus services and this list is growing every new week now that would make you wonder how aws became so successful of course it's their customers let's talk about the list of well-known companies that has their idea environment in aws adobe adobe uses aws to provide multi-terabyte operating environments for its customers by integrating its system with aws cloud adobe can focus on deploying and operating its own software instead of trying to you know deploy and manage the infrastructure airbnb is another company it's an community marketplace that allows property owners and travelers to connect each other for the purpose of renting unique vacation spaces around the world and the airbnb community users activities are conducted on the website and through iphones and android applications airbnb has a huge infrastructure in aws and they're almost using all the services in aws and are getting benefited from it another example would be autodesk autodesk develops software for engineering designing and entertainment industries using services like amazon rds or rational database service and amazon s3 or amazon simple storage servers autodesk can focus on deploying or developing its machine learning tools instead of spending that time on managing the infrastructure aol or american online uses aws and using aws they have been able to close data centers and decommission about 14 000 in-house and co-located servers and move mission critical workload to the cloud and extend its global reach and save millions of dollars on energy resources bitdefender is an internet security software firm and their portfolio of softwares include antivirus and anti-spyware products bitdefender uses ec2 and they are currently running few hundred instances that handle about five terabytes of data and they also use elastic load balancer to load balance the connection coming in to those instances across availability zones and they provide seamless global delivery of servers because of that the bmw group it uses aws for its new connected car application that collects sensor data from bmw 7 series cars to give drivers dynamically updated map information canon's office imaging products division benefits from faster deployment times lower cost and global reach by using aws to deliver cloud-based services such as mobile print the office imaging products division uses aws such as amazon s3 and amazon route 53 amazon cloudfront and amazon im for their testing development and production services comcast it's the world's largest cable company and the leading provider of internet service in the united states comcast uses aws in a hybrid environment out of all the other cloud providers comcast chose aws for its flexibility and scalable hybrid infrastructure docker is a company that's helping redefine the way developers build ship and run applications this company focuses on making use of containers for this purpose and in aws the service called the amazon ec2 container service is helping them achieve it the esa or european space agency although much of esa's work is done by satellites some of the programs data storage and computing infrastructure is built on amazon web services esa chose aws because of its economical pay as ego system as well as its quick startup time the guardian newspaper uses aws and it uses a wide range of aws services including amazon kinesis amazon redshift that power an analytic dashboard which editors used to see how stories are trending in real time financial times ft is one of the world's largest leading business news organization and they used amazon redshift to perform their analysis a funny thing happened amazon redshirt performed so quickly that some analysis thought it was malfunctioning they were used to running queries overnight and they found that the results were indeed correct just as much faster by using amazon redshift fd is supporting the same business functions with costs that are 80 age lower than what was before general electric ge is at the moment as we speak migrating more than 9000 workloads including 300 desperate erp systems to aws while reducing its data center footprint from 34 to 4 over the next three years similarly howard medical school htc imdb mcdonald's nasa kellogg's and lot more are using the services amazon provides and are getting benefited from it and this huge success and customer portfolio is just the tip of the iceberg and if we think why so many adapt aws and if we let aws answer that question this is what aws would say people are adapting aws because of the security and durability of the data and end-to-end privacy and encryption of the data and storage experience we can also rely on aws way of doing things by using the aws tools and techniques and suggested best practices built upon the years of experience it has gained flexibility there is a greater flexibility in aws that allows us to select the os language and database easy to use swiftness in deploying we can host our applications quickly in aws beat a new application or migrating an existing application into aws scalability the application can be easily scaled up or scaled down depending on the user requirement cost saving we only pay for the compute power storage and other resources you use and that too without any long-term commitments now let's talk about the different types of services that aws provides the services that we talk about fall in any of the following categories you see like you know compute storage database security customer engagement desktop and streaming machine learning developers tools stuff like that and if you do not see the service that you're looking for it's probably is because aws is creating it as we speak now let's look at some of them that are very commonly used within computer services we have amazon ec2 amazon elastic bean stock amazon light sale and amazon lambda amazon ec2 provides compute capacity in the cloud now this capacity is secure and it is resizable based on the user's requirement now look at this the requirement for the web traffic keeps changing and behind the scenes in the cloud ec2 can expand its environment to three instances and during no load it can shrink its environment to just one resource elastic beanstalk it helps us to scale and deploy web applications and it's made with a number of programming languages elastic beanstalk is also an easy to use service for deploying and scaling web applications and services deployed a bit in java.net php node.js python ruby docker and lot other familiar services such as apache passenger and iis we can simply upload our code and elastic beanstalk automatically handles the deployment from capacity provisioning to load balancing to auto scaling to application health monitoring and amazon light sale is a virtual private server which is easy to launch and easy to manage amazon lightsail is the easiest way to get started with aws for developers who just need a virtual private server lightsail includes everything you need to launch your project quickly on a virtual machine like ssd based storage a virtual machine tools for data transfer dns management and a static ip and that too for a very low and predictable price aws lambda has taken cloud computing services to a whole new level it allows us to pay only for the compute time no need for provisioning and managing servers and aws lambda is a compute service that lets us run code without provisioning or managing service lambda executes your code only when needed and scales automatically from few requests per day to thousands per second you pay only for the compute time you consume there is no charge when your code is not running let's look at some storage services that amazon provides like amazon s3 amazon glacier amazon abs and amazon elastic file system amazon s3 is an object storage that can store and retrieve data from anywhere websites mobile apps iot sensors and so on can easily use amazon s3 to store and retrieve data it's an object storage built to store and deter any amount of data from anywhere with its features like flexibility and managing data and the durability it provides and the security that it provides amazon simple storage service or s3 is a storage for the internet and glacier glacier is a cloud storage service that's used for archiving data and long term backups and this glacier is an secure durable and extremely low-cost cloud storage service for data archiving and long-term backups amazon ebs amazon elastic block store provides block store volumes for the instances of ec2 and this elastic block store is highly available and a reliable storage volume that can be attached to any running instance that is in the same availability zone abs volumes that are attached to the ec2 instances are exposed as storage volumes that persistent independently from the lifetime of the instance an amazon elastic file system or efs provides an elastic file storage which can be used with aws cloud service and resources that are on premises and amazon elastic file system it's an simple it's scalable it's an elastic file storage for use with amazon cloud services and for on-premises resources it's easy to use and offers a simple interface that allows you to create and configure file systems quickly and easily amazon file system is built to elastically scale on demand without disturbing the application growing and shrinking automatically as you add and remove files your application have the storage they need and when they need it now let's talk about databases the two major database flavors are amazon rds and amazon redshift amazon rds it really eases the process involved in setting up operating and scaling a relational database in the cloud amazon rds provides cost efficient and resizable capacity while automating time consuming administrative tasks such as hardware provisioning database setup patching and backups it sort of frees us from managing the hardware and sort of helps us to focus on the application it's also cost effective and resizable and it's also optimized for memory performance and input and output operations not only that it also automates most of the services like taking backups you know monitoring stuff like that it automates most of those services amazon redshift amazon redshift is a data warehousing service that enables users to analyze the data using sql and other business intelligent tools amazon reshift is an fast and fully managed data warehouse that makes it simple and cost effective and lays all your data using standard sql and your existing business intelligent tools it also allows you to run complex analytic queries against petabyte of structured data using sophisticated query optimizations and most of the results they generally come back in seconds all right let's quickly talk about some more services that aws offers there are a lot more services that aws provides but are we going to look at some more services that are widely used aws application discovery services help enterprise customers plan migration projects by gathering information about their on-premises data centers in a planning a data center migration can involve thousands of workloads they are often deeply interdependent server utilization data and dependency mapping are important early first step in migration process and this aws application discovery service collects and presents configuration usage and behavior data from your servers to help you better understand your workloads route 53 it's a network and content delivery service it's an highly available and scalable cloud domain name system or dns service and amazon route 53 is fully compliant with ipv6 as well elastic load balancing it's also a network and content delivery service elastic load balancing automatically distributes incoming application traffic across multiple targets such as amazon ec2 instance containers and ip addresses it can handle the varying load of your application traffic in a single available zones and also across availability zones away auto scaling it monitors your application and automatically adjusts the capacity to maintain steady and predictable performance at a lowest possible cost using aws auto scaling it's easy to set up application scaling for multiple resources across multiple services in minutes auto scaling can be applied to web services and also for db services aws identity and access management it enables you to manage access to aws services and resources securely using iam you can create and manage aws users and groups and use permissions to allow and deny their access to aws resources and moreover it's a free service now let's talk about the future of aws well let me tell you something cloud is here to stay here's what in store for aws in the future as years pass by we're going to have a variety of cloud applications bond like iot artificial intelligence business intelligence serverless computing and so on cloud will also expand into other markets like health care banking space automated cars and so on as i was mentioning some time back lot or greater focus will be given to artificial intelligence and eventually because of the flexibility and advantage that cloud provides we're going to see a lot of companies moving into the cloud all right let's now talk about how easy it is to deploy and web application in the cloud so the scenario here is that our users like a product and we need to have a mechanism to receive input from them about their likes and dislikes and you know give them the appropriate product as per their need all right though the setup and the environment it sort of looks complicated we don't have to worry because aws has tools and technologies which can help us to achieve it now we're going to use services like route 53 services like cloudwatch ec2 s3 and lot more and all these put together are going to give an application that's fully functionable and an application that's going to receive the information like using the services like route53 cloudwatch ec2 and s3 we're going to create an application and that's going to meet our need so back to our original requirement all i want is to deploy a web application for a product that keeps our users updated about the happenings and the new comings in the market and to fulfill this requirement here is all the services we would need ec2 here is used for provisioning the computational power needed for this application and ec2 has a vast variety of family and types that we can pick from for the types of workloads and also for the intents of the workloads we're also going to use s3 for storage and s3 provides any additional storage requirement for the resources or any additional storage requirement for the web applications and we are also going to use cloudwatch for monitoring the environment and cloudwatch monitors the application and the environment and it uh provides trigger for scaling in and scaling out the infrastructure and we are also going to use route 53 for dns and route 53 helps us to register the domain name for our web application and with all the tools and technologies together all of them put together we're going to make an application a perfect application that caters our need all right so i'm going to use elastic bean stock for this project and the name of the application is going to be as you see gsg sign up and the environment name is gsg signup environment 1. let me also pick a name let me see if this name is available yes that's available that's the domain name so let me pick that and the application that i have is going to run on node.js so let me pick that platform and launch now as you see elastic beanstalk this is going to launch an instance it's going to launch the monitoring setup or the monitoring environment it's going to create a load balancer as well and it's going to take care of all the security features needed for this application all right look at that i was able to go to that url which is what we gave and it's now having an default page shown up meaning all the dependencies for the software is installed and it's just waiting for me to upload the code or in specific the page required so let's do that let me upload the code i already have the code saved here that's my code and that's going to take some time all right it has done its thing and now if i go to the same url look at that i'm being thrown an advertisement page all right so if i sign up with my name email and stuff like that you know it's going to receive the information and it's going to send an email to the owner saying that somebody had subscribed to your service that's the default feature of this app look at that email to the owner saying that somebody had subscribed to your app and this is their email address stuff like that not only that it's also going to create an entry in the database and dynamodb is the service that this application uses to store data there's my dynamodb and if i go to tables right and go to items i'm going to see that a user with name samuel and email address so and so has said okay or has shown interest in the preview of my site or product so this is where this is how i collect those information right and some more things about the infrastructure itself is it is running behind and load balancer look at that it had created a load balancer it had also created an auto scaling group now that's the feature of elastic load balancer that we have chosen it has created an auto scaling group and now let's put this url you see this it's not a fancy url right it's an amazon given url a dynamic url so let's put this url behind our dns let's do that so go to services go to route 53 go to hosted zone and there we can find the dns name right so that's a dns name all right all right let's create an entry and map that url to our load balancer right and create now technically if i go to this url it should take me to that application all right look at that i went to my custom url and now that's pointed to my application previously my application was having a random url and now it's having a custom url choose from over 300 in-demand skills and get access to 1 000 plus hours of video content for free visit scale up by simply learn click on the link in the description to know more cloud services are now available to satisfy almost any id requirement although cloud computing services vary greatly they all have some basic qualities and benefits in common and they can be classified into a few basic cloud service kinds hello everyone i am sharmly and i welcome you all to this new video of simply learn on cloud computing services let me give you an overview of the concepts you are going to go through in this video first we will go through what is cloud computing then we will have an introduction to cloud computing services followed by the types of cloud computing services and at the end we will look into the features and benefits of cloud computing services so let's not waste any more time and get started with the video so what is cloud computing cloud computing is the distribution of cloud computing services by the internet including servers storage databases networking software analytics intelligence to provide faster innovation more flexible resources and economies of scale you usually only pay for the cloud services you use which helps you cut cost run your infrastructure more efficiently and scale as your business grows now that we know what cloud computing is let's proceed with understanding cloud computing services cloud computing services provide users with a variety of capabilities including email storage backup data retrieval app creation testing data analysis audio video streaming software on demand regardless of the type of service although cloud computing is still a relatively new technology it is now being used by a wide range of organizations including large enterprises small businesses non-profit government agencies and even individual consumers so not all clouds are created equal and not every sort of cloud computing is appropriate for every situation a variety of models varieties and services have evolved to assist you find the best option for your needs to begin one must decide on the type of cloud deployment or cloud computing architecture that will be used to implement your cloud services cloud services can be deployed in three different ways public cloud private cloud or hybrid cloud now let's dive down deeper into cloud computing services and explore its type in more detail cloud computing unlike a microprocessor is not a single piece of technology rather it's a system made up of three services infrastructure as a service platform as a service and software as a service so let's have a better understanding of each starting with infrastructure as a service so what is infrastructure as a service it is a type of cloud computing that uses the internet to provide virtualized computing resources the cloud provider controls id infrastructures such as storage server and networking resources and also offers them to subscribe companies via virtual machines accessible over the internet in ias model for businesses it can provide numerous advantages including the ability to make tasks faster easier more flexible and less expensive now let's have a look at the working of iaas users connect to resources and services across a wide area network such as the internet and then use the cloud provider services to complete the application stack the user can for example log into the infrastructure as a service platform to build virtual machines install operating systems in each vm deploy middleware such as databases create storage buckets for workloads and backups and install the enterprise workload onto that virtual machine customers can then track cost monitor performance balance network traffic solve application difficulties and manage disaster recovery using the provider's service so moving ahead let's go through its advantages and disadvantages advantages organizations select infrastructure as a service because operating a workload without having to buy manage and support the underlying infrastructure is frequently easier faster and more cost effective a company can rent or lease infrastructure from another company using infrastructure as a service for workloads that are transitory experimental or change abruptly it is an effective cloud service paradigm for example if a company is creating a new software product hiring an ieas provider to host and test the application may be more cost effective once the new software has been thoroughly tested and refined the company can move it away from the iaas environment and into a more traditional in-house deployment if the expenses of a long-term commitment are lower the organization could commit base or software to a long-term iaas deployment now disadvantages billing can be a challenge for some firms despite its flexible pay-as-you-go model cloud invoicing is quite detailed and it is broken down to reflect specific service usage when evaluating the invoices for each resource and service involved in application deployment users frequently experience sticker shock or discover expenses that are greater than expected another issue it face is lack of insight because its providers own the infrastructure the configuration and performance of that infrastructure are very rarely transparent to its con consumers users may find it more difficult to operate and monitor systems due to the lack of transparency users of infrastructure as a service are also concerned about service availability and reliability the supplier has a big influence on the workload's availability and performance the workloads of users will be impacted if a provider has network constraint or any other type of internal or external outage furthermore because it is a multitude in design the problem of noisy neighbors might have an adverse effect on users workloads so now these are the top infrastructure as a service providers linode is a privately held cloud hosting firm base in the united states that offers virtual private servers host winds web hosting cloud hosting and dedicated server options are all available from host wins microsoft azure is a cloud computing service operated by microsoft for application management via microsoft managed data centers digitalocean offers developers cloud service that make it easy to deploy and grow programs that run on several machines at the same time alibaba cloud is a cloud computing firm that serves online businesses as well as alibaba's own e-commerce ecosystem so the second service is platform as a service what is platform as a service platform as a which are geared towards software development teams include computing and storage infrastructure as well as development platform layer that includes web servers database management systems and software development kits for multiple programming languages working of platform as a service for software development it does not replace an organization's complete ip infrastructure it's made possible by the hosted infrastructure of a cloud service provider a web browser is the most common way for users to access the offsets platform as a service such as application hosting and java development can be supplied by a public private or hybrid clouds so now let's look into the advantages and disadvantages advantages the main advantage of platform as a service for users is its simplicity and convenience much of the infrastructure and other id services will be provided by the platform as a service provider which users can access from anywhere via a web browser the flexibility to pay on a per use basis allows businesses to forego the capital cost associated with on-premises gear and software many platform-as-a-service solutions are aimed towards software developers these platform provider computation and storage infrastructures as well as text editing version management compilation and testing capabilities to assist developers in swiftly and efficiently developing your software coming to the disadvantages platform as a service on the other hand can cause issues with service availability and resilience customers may suffer as a result of of a service outage or other infrastructure interruption which might result in costly productivity losses its suppliers on the other hand will typically deliver reasonably high up times another widespread problem is when the lock-in which occurs when users are unable to simply transition many of their services and data from one platform as a service solution to another when choosing a provider users must consider the business risk of service outages and vendor login internal changes to a platform as a service product could also be a problem the impact on users might be tough and restrictive if a platform as a service provider seizes supporting a programming language or chooses to use a different set of development tools so let's have a look at the top platform as a service provider ibm cloud computing refers to a set of cloud computing services for businesses provided by ibm a technology corporation red hat is a collection of tightly integrated red hat technologies for building cloud infrastructure and developing cloud native apps on premises elastic beanstalk aws elastic beanstalk is an amazon web services application deployment orchestration solution salesforce offers crm services as well as enterprise applications for customer service marketing automation analytics and application development and software ag cloud is an open and independent cloud platform that serves as your one stop shop for all software ag has to offer in the cloud now coming to the next service which is software as a service so what is software as a service customer relationship management or crm marketing automation and business analytics are just a few of the application level services offered by software as a service companies so how does it work the provider gives consumers network access to a single copy of an application that the provider designed expressly for software as a service distribution in the software on demand model the source code for the program is the same for all clients and new features or functionalities are rolled out to all the users at the same time the data of each model's customer may be stored locally in the cloud or both locally in the cloud depending on the service level agreement coming to its advantages customers subscribe to a software as a service solution rather than buying and installing software or additional gear to support it many firms can now budget more effectively and predictably by converting costs to recurrent operating expenses you can also cancel software as a service subscriptions at any time to avoid incurring recurring fees its systems are frequently customizable and can be connected with other corporate applications particularly when using software for the same vendor vertical scalability is a feature of cloud services like software as a service which allows clients to access more or fewer services or features on demand disadvantages of software as a service when providers endure service delays impose undesirable modifications to service offerings or suffer a security breach all of these things can have a significant impact on customers ability to use the software as a service offering customers should be aware of their software as a service provider's sla and ensure that it is followed if the provider adopts a new version of an application it will roll it out to all of its clients whether or not they want it this may need the organization allocating additional training time and resources switching vendors can be tough as it is with any cloud service provider customers must migrate massive volumes of data when switching vendors furthermore some vendors use technologies and data types which can make transferring client data between cloud providers even more difficult so coming to the top companies that provide software as a service adobe is a collection of adobe incorporation programs and services that provide user with access to software for graphic design video editing web development and photography as well as a set of mobile apps and certain optional cloud services sap is a platform as a service designed by sap se for developing new application and enhancing existing ones in a secure cloud computing environment managed by sap google cloud google cloud platform is a set of cloud computing services provided by google that run on the same infrastructure as google's internal products such as google search gmail drive and youtube fresh works cloud platform allows users to manage their identities and access across all of their freshworks products and athelation advanced features such as a 99.9 uptime sla unlimited storage and premium support provide teams the confidence to scale reliably now that we have all the in-depth information about different types of cloud computing services let's move ahead and explore its benefits and features which makes it so popular and convenient first the supplier hosts and maintains the site in their own facility the cloud hosting provider acquires hosts and maintains the necessary hardware and software users avoid the construction expenditures and maintenance problem that would be incurred if the service was established on premise second self-service using a web-based interface through a web interface service users can initiate certain service functions as well as increase or reduce their service consumption level with little or no intervention from the service provider third is you must pay to utilize the facility users of the service only pay for the services they use when compared to the typical method of building on-site id capacities targeted for the highest usage situations and then having that capacity to go unused for the most of the time this can result in significant cost savings the fourth one is scalability that comes close to being infinite cloud computing service providers usually have the instruct infrastructure in place to deliver their services at a large scale that means that cloud service consumers can readily accommodate business expansion or periodic surges in service usage well this is all about cloud computing services hello everybody welcome to simply learn's aws s3 tutorial for beginners my name is kent and today i'm going to be covering these following points i'm going to be covering what is cloud computing i'm going to be showcasing what aws is in terms of cloud computing and i'm also going to be covering the core fundamental service of the simple storage service which is a object storage service we're going to be covering the benefits of the simple storage service better known as s3 what objects and buckets are and we're going to be seeing how things are working in the background and implementing that in terms of lab assisted demonstrations in order for you to see what those features of s3 are and then we'll perform a wrap up of the material we've done in a conclusion so let's get started what is cloud computing so here is a definition of cloud computing but i'm going to paraphrase it cloud computing is nothing more than you gaining access to infrastructure through web services with a pay-as-you-go model now this is a very very different model than where traditionally involved in on a on-prem data center so when i mean on-prem i'm saying on-premise okay so you'll hear that a lot on-prem now when we are on on-premise data center we have to provision lots of infrastructure even before we get started with um deploying any type of application whether it be a mobile application a web application etc etc so every application needs some sort of underlying infrastructure whether that be bare bone servers databases they're obviously going to be needing some types of storage etc etc so all this is something that we have to kind of fight against in order just to get started with deploying our application so this takes usually traditionally depending on the size of the company about three months to set up so what if we could leverage all this infrastructure as a service and just through some api call that's what the web services is all about we could provision a server or we could provision a database within minutes and then deploy any type of application that we want to on top of that stack and also take advantage of any storage that we may want we may want object level storage which is what we're going to cover we may want an elastic file system right so there are different types of storages that we're eventually going to look at as you go down your learning path with simply learn so cloud computing is really about provisioning of all these kinds of services but not only at the infrastructure level we're going to see how we could move up the stack and get entire platforms as a service or even softwares as a service so what is aws well aws is really a cloud computing platform that will offer us many many services through these api calls some are again just bare infrastructure like a service other can be a service as a software for example email so here we have an example of the public data storage this is s3 the simple storage service icon that you're going to see over and over again this is a representation of an actual bucket in s3 so this will become more clear as we continue the slide deck and actually reinforce this with some hands-on lab but as you can imagine you may want to provision a private software defined network in the cloud and so we can do that in aws we can provide load balancing we could provide scalability in terms of auto scaling groups that will respond to increased demand as your application let's say becomes more popular you want that infrastructure to grow elastically so doing this on premise is traditionally not only extremely expensive and thus prohibited but very difficult to implement physically speaking as well so by going with a cloud provider like aws we can severely reduce the expense and the complexity of setting all of this up because we're trading capital expense for variable expense so capital expense is about you procuring all the hardware that you need beforehand and the budget being approved before even getting that hardware in through your door right and then you have to configure it so there's that complexity factor whereas if you're doing this through aws you can within minutes just provision all of that very quickly so you're also not paying for everything all up front you're paying this with a pay as you go model so very different we are now removing what's called undifferentiated lifting which is everybody has to do the same thing but it's not really bringing any value to their end product we all have to buy servers and configure them and scale them etc etc so if we remove that element and have aws handle that we can concentrate on our business which is our application and improving that application and responding more to our clients request we can be more agile more flexible so all of this leads to these points that we see here on the slide much easier to use much easier to get our application to market we're actually going to be leveraging the physical security that aws implements for us they have data centers all over the world that we're going to plug into and that physical security is already implemented for us there are varying levels of security that we're going to get into through our aws journey but right off the bat you know that you have physical security of course we're going to be talking about storage in this demonstration or in this slide deck over here but the nice thing is is we only pay for what we use so that's that variable expense that i was talking about that we've traded for capital expense we can easily set up databases we can easily automate our backups and have those backups stored in the cloud through that s3 storage that we're going to get into okay so lots of benefits of going through aws here and it's also comprehensive and simple to use uh comprehensive meaning that there are hundreds and hundreds of services um it's going to be actually quite time consuming for you to just know what each service does in general but you'll get there through your journey it's all about sticking with it and learning something new every day which is what simple learn is all about over here we have infrastructure as a service platform as a service and software as a service so these are three different types of distinctions we have in the cloud industry to basically cover what's going on here on the right hand side so on premise you're traditionally involved in everything from networking which is you for example buying the switches the routers the cabling everything all the way up the stack to deploying your application and you see everything in between there okay so this is a very time consuming job not only that but it costs a lot of money because you have to buy all these machines up front and you have to have the individuals with the right knowledge to maintain all of this stack all right so different individuals in different teams maintaining this whole on-premise data center you may also have different data centers deployed globally so you can imagine the cost overhead of this so everything in green is basically you're managing it now we can offset that and ask aws to handle part of this stack over here so on this second column over here we see infrastructure as a service which basically says aws i want you to handle all the networking my software defined network i want you to handle my storage elasticity i want you to handle my compute power my virtual machines i want you to handle even administrating those machines at a physical layer as well and i want you to physically secure them so this will allow you more time to focus on all the green up here of course we can move up the stack and say i even want aws to take on more responsibility and that would be more of a platform as a service where aws would also install and maintain the operating system for you so you can imagine things like new versions of operating systems uh security patches maintenance patches all of the like and any middleware or runtime that you have on top of that think of something like the java a virtual machine perhaps that needs to be updated now you wouldn't be responsible for maintaining all of what's in orange over here you would only be responsible for deploying your application and making sure that your data is saved and secured right so again platform as a service is a more hands-off approach so think of it as you wanted to set up a cicd deployment environment for your development team and you want it to be very uh involved in just handling your application code and making sure that it was properly compiled built tested and deployed right well this could be implemented as a platform as a service or you could take your code as a zip file and give it to aws's service called beanstalk which would automatically deploy your code and make sure that it's highly available scalable and fault tolerant across all of the orange stack over here right so more of a hands-off approach of course we can go all the way up the stack and tell aws to take care of everything for us and just ask for a service much like a utility like electricity or water we might want to say we just want an email software service and we're not really concerned about how the underlying operating system or servers or network are configured for that we just want to use the software as a service we don't want to administrate anything else we just want to be sort of like an end user to it so aws will manage everything underneath the hood so they're varying types of services you pick and choose whichever one makes more sense for you and your team so what is s3 s3 stands for simple storage service and it is an object storage service in amazon what this means is that it doesn't matter what kind of file you upload to s3 it will be treated as an abstraction meaning an object so you can upload a pdf file you can upload a jpeg file you can upload a database backup it doesn't really matter all that is abstracted away by the use of storing it within an object we're going to talk more about what composes an object but by doing so what happens is s3 allows us to have industry leading scalability so it doesn't matter if you're going to make a request for that database backup let's say that object um five times or ten thousand times that request that demand will scale the underlying infrastructure that is implemented behind the scenes is handled for you so in a sense it's kind of like a serverless service a storage service where you're not implicated in handling anything underneath the covers you're just uploading your objects and accessing them later on in terms of data availability very powerful as well because it takes those objects and will replicate them across at least three availability zones of course these availability zones have one or more data centers attached to them so you have lots and lots of copies of your objects distributed globally or at least at a regional in a regional area you can do this as well globally if you enable global replication and you will see that your level of juror availability will skyrocket and you're just not going to worry about losing an s3 object data security well we can encrypt our data at rest our objects at rest we can encrypt it in transit we can also come up with security policies at the bucket level which we're going to talk about what a bucket is very very very soon these are going to be implemented through what's called i am policies and you're going to be able to control who or what has access to your objects and of course because everything is handled underneath the covers all the servers all the storage nodes are cloud optimized for the best possible performance so let's continue on our journey on what is s3 we're going to take a look at what is a bucket and what is an object as you can see here you have inside this bucket which is a logical container for an unlimited amount of objects you could have objects of different shapes and sizes like i said you could have pictures database backups etc so this is what's being represented here by different shapes and you can think of a bucket really as almost like a folder where objects or files are placed within them so there are many ways for us to place objects within a bucket we can do this through the aws console which i'll be showing you very shortly we can do this via the command line interface or we can do this to a software development kit at the end of the day all those three options go through an api right so it's all about picking the right method that is best suited for whatever kind of end user or application that needs access to these buckets and to these objects now once you've got the data or objects within this bucket you could control pretty much how the data is accessed how it's stored whether you want it to be encrypted and how it's even managed so you can have organizations that have specific compliance needs for example any objects that are placed in or perhaps some pdfs are not to be modified or not to be touched for 90 days let's say we can employ object locks if we want to we can imply security guards we can also for example record all api actions that try to access to list to delete any kind of api operations on those objects at the bucket level or on the object level we can record if that is some sort of organizational compliance need for auditing or for whatever internal auditing reason that you may have so continuing on here you can see that there are many organizations that use s3 one in point is amazon itself s3 is so popular and so durable that amazon internally uses it to store its own data you're guaranteed you know almost that you're not going to lose any object in there because it has what we call 11 9 durability now 11 9's durability is really an extreme amount of durability where it's mathematically almost impossible for you to lose an object once you place it in s3 you're in fact more liable to get hit by meteorite than you are to lose an object in s3 statistically speaking which is a pretty incredible statistic but it's but it's true so if we continue here with the benefits of s3 some of these have already described but let's talk about the full picture here we see the performance scalability availability and durability of s3 are really you know first class in the industry again durability we were talking about that 11 9 so what that really means is 99.9999 in total if you count all the nines that's 11 of those nine so um ex most durable in the industry by far and again because everything is handled underneath the covers it's a serverless service they ensure the scalability and availability and the performance of the inner workings of that object level storage now cost is of course always important and s3 really shows up here with first class support again for cost it's got very very low cost we can have object level storage for let's say a terabyte of object level storage for as little as a dollar a month so again it will depend on the kind of storage class we're going to have a discussion on different types of storage classes that we might want to transition or move over our objects from one storage class to the next depending on how frequently accessed that data is as you know data over time seems to get less and less accessed as you know your needs shift from one place to another we're going to talk about that coming up very soon so we're going to want to really focus on that to reduce our end of the month costs for storage with s3 of course security is always at the forefront and we want to make sure that we either simply secure by encrypting everything right off the bat either at rest in transit or we want to put an object lock or just maintain security at the bucket level maintaining let's say who or what has access to that data because by default no one has access to the data unless we open up the door and also we want to make sure that we don't give public access to our bucket for obvious reasons of giving away important information by mistake so aws makes it extremely difficult for you to accidentally do this and i'm going to show this in a demonstration lab coming up very soon and we can also query in place which is very interesting so you can imagine you putting let's say a file that's in csv format or json format or parquet format some sort of structured format or semi-structured format in s3 and you can actually use sql queries on the spot to filter that data in place at the bucket level you could also have other services that may want to extract some business intelligence from that data in your s3 bucket directly as well so some other more advanced querying operations can take place at the s3 level so this you will learn during your journey with simply learn as you're covering all the aws technology stacks and here the most widely used storage cloud service in the industry because of all the points that we just covered it really is the number one storage or object level storage solution in the industry let's now take a look at objects and buckets and s3 so the objects are really the fundamental entities that are stored in s3 or the lowest common denominator which means that we're not really interested in what kind of data is within the object at the layer of s3 because s3 doesn't have direct access to the data within an object like i said before it's an abstraction of the data so we don't know if it's a cat's picture if it's a backup of your database it's just treated as a fundamental entity aka the object now every object is associated with metadata so data about itself things like what's the name of the object what's the size the time and date that the object was uploaded things of that nature are categorized as metadata so s3 does have direct access to that now the data within that object of course is accessible by other services so it's not that once you've uploaded it you've totally lost the data and it's only can be treated as an object it's just that at this layer it's simply just assigned metadata and a version id a unique version id and if you re-upload the exact same object the second or third time to s3 it will have its own version id number so a new unique version id number will be generated so really what buckets are are there logical containers to store those objects so of course at an extremely high level a bucket would be like your root file system if you want to think about it like that but that doesn't mean you can't go into this bucket and create separate folders now when you create separate folders in a bucket because you might want to logically organize all your objects you might be fooled by the fact that you think this is a hierarchical storage system when in fact it is not and i'll talk about that in a second so you cannot store an object without first storing it into a bucket so of course the first step would be for us to create a bucket and then upload objects within that bucket so an object cannot can cannot exist without its container or its bucket so there's no windows explorer view like we're used to in an operating system because this is not a hierarchical view no matter if you create folders within folders within folders in fact s3's internal architecture is a flat hierarchy what we do instead is we assign prefixes which are treated as folders in order to logically organize our objects within our buckets and i'm going to showcase a prefix in one of the demonstrations coming up very soon so when you create or when you're working with s3 first of all you're working at a regional level you're gonna have to pick a region for example you might pick us east one region or usc ii region and the bucket that you're going to be creating will be replicated across several availability zones but within that region also that data those objects can be accessed globally because s3 uses the http protocol http protocol is very permissive everybody knows how to administrate it and work with it so we just need to give access to that object in terms of a permission policy and give it the proper url or give whoever needs access to it the proper url and they can access that via http globally so first when you're creating a bucket you have to select the region and the object will live within that region but that doesn't mean that it still can't be accessed globally so let's now take a minute go over to the aws console and actually let me showcase how to create objects and buckets so for our first lab we're going to be going and creating an s3 bucket and uploading an object to it so we first have to log into the aws console which we have up here in the address bar let's click on either create free account if it's your very first time or as in is in my case i already have an account we're going to sign in here you're going to have to pass in your im username password and your account id of course once you've logged in you can search for the simple storage service s3 either by coming up here in the search box and typing s3 and you'll find it there that's probably the easiest way or if you want you can take a look at all the services and it'll be under storage over here the first one okay so pick whichever method you see best for yourself once we're here we want to create our first bucket now i've already have a couple of buckets here so we're gonna go on and create a new bucket the very first thing is you have to specify a bucket name now this bucket name has to be globally unique if you take a look here at the region that we're in it doesn't actually select a specific region it selects the global option which means that this bucket will become globally accessible so that is why it needs to have a unique name much like a dns name has to be unique for your website right so i'm going to come up here and pick a name that i think is going to be unique so i'm going to say simply learn s3 demo all right let's take a look at that's going to work out of course we have to pick a region but it is globally accessible so you either pick a region that's closest to your end users or in our case since we're just doing a demonstration we can do whichever is closer to us right now okay and we're going to skip these options for now we're going to come back to them later on we're just going to create the bucket now hopefully that was a unique name and allowed me to create that and it looks like it did so if i scroll down you can see that it clearly got created over here now we're going to click on that and we're going to start uploading objects to this bucket so let me click on the upload button and you can either select one or more files or you can select an entire folder so i'm going to just go and select a specific cat picture that i have here okay and again we'll go through some of those other options later on and we'll just click upload now this should take no time at all because of the fact that it's a very small object or file that's being uploaded so it is has succeeded we can close this up and we see now clearly that the object is in our bucket if we go to properties we can see the metadata associated with this bucket we can see the region that it's in we can see the arn which is the amazon resource name which uniquely identifies this resource this bucket across the globe so if ever you needed to reference this let's say in an im policy or whatever other service needed to communicate with s3 you would need this arn it's very important piece of information and of course we have the creation date so some high level metadata so objects as we have covered already consist of not only the data itself but the metadata so there's lots of metadata and there's a lot of other features that we can go here and enable very easily and this will be the basis of the future demonstrations that i'm going to do all right so just to recap what we just did we created a unique bucket give it a name of simply learn s3 demo and uploaded our first object to it so let's now take a look at the inner workings of the amazon s3 when we upload an object into a bucket we have to select which one of these storage classes the object will reside in so you see have six storage classes here at the bottom and each have their own characteristics that we're going to get into by default if you don't specify anything it'll get placed in what's called the s3 standard storage class which is the most expensive out of all the storage classes once your object gets colder and what i mean by colder is your axis patterns diminish meaning that you're accessing that file less and less over the course of time so it gets colder you will transition that object from one tier to the next all the way to for example s3 deep archive so again deep archive signifies extremely cold so maybe you're only referencing this data once a year once every couple of years and so you want to have the cheapest possible storage available so right now you can get about one terabyte of storage per month for about a dollar a month with s3 glazer deep archive so you are going to be very interested in knowing how to transition from one or more of these storage classes over time in order to save on your storage costs that's really why we're doing this so let's go through some of the storage classes by default like i said whenever you upload an object you automatically get placed into the standard storage class so any files that you're working on frequently daily this is the best fit for it you've got the highest level of accessibility and um durability as well not that the others don't have the same level of durability however we'll see how when you transition from one storage tier to the next some characteristics do change in order for you to save on some cost we're going to go through some of those now this would be considered hot data right data that's used all the time maybe just by you maybe by everybody right so that's the perfect place to place it in the standard storage class now over time like i said you may find yourself working less and less perhaps on a document that was due by the end of the month that document was submitted and then afterwards you don't work on that document anymore perhaps you're only working on revisions based on feedback from your colleagues that are asking you to make some corrections or some amendments and so only those corrections or amendments come in perhaps once a month and so in that case you might find yourself finding a justification for moving that document from the standard tier to the standard ia or infrequently accessed tier maybe any objects not modified for more than 30 days are a good fit for that and that's really the criteria for ia in active access is that s3 or aws itself recommends to only put objects in there if they haven't been asked access for at least 30 days so you get a a price reduction a rebate for putting objects that are not accessed frequently in here of course if you remove objects let's say before the 30 day limit then you are charged a surcharge for retrieving an object that you said was infrequently accessed but it really was not so bear that in mind if you're going to place objects in infrequent access be somewhat reasonably assured that you're not going to be going there and accessing them you can still access those files no problem just as quickly they have the same level of accessibility and durability however like i said anything less than 30 days you'll get a price thing on that if you want to have long-term storage we're talking about amazon glacier so this is more anything over 90 days that hasn't been modified or 180 days there's two subcategories of amazon glacier that we're going to get into and this is the cheapest storage by far and amazon glacier doesn't really operate through the aws console as the same as the standard and the infrequent access you can't really upload objects to glacier via the console in the browser you can only do so let's say through the command line interface or through an sdk the only thing you can do on the web console with glacier is actually create what's called a vault which is a logical container for your archives but then after that you have to go through the cli or the sdk to do the rest of the work there if we continue on there's some gray areas between uh the s3 standard and the glacier one is the one zone in frequently accessed storage class so if we go back to the regular standard and ia storage class all of these objects are stored across a minimum of three availability zones if you want a further price reduction you can store your objects in a one zone ia storage class which means that instead of taking that object and replicating it across three or more availability zones it will only store it in a single availability zone therefore reducing the level of availability that you have to that object so in this case here if that single availability zone would go down for example you would not have access to that object once it would come back up of course you would the other thing is is if there was a an immense catastrophe where the actual availability zone was destroyed well of course then your object is also gone so if that's something that doesn't worry you because you have already many copies of this object maybe lying around on premise then this is a good option for you because it's data that you're willing to lose or lose access to for short periods of time if ever that single availability zone goes down so it's about an extra 20 off the price from already the normal ia standard price there is another one called the standard reduced redundancy storage this one is kind of getting phased out as we speak because the same price for this storage class is about the same amount you're going to pay for the normal ia standard class what this does is again is a good fit for your objects that you're not really worried about losing if there is some sort of catastrophe that happens in an availability zone there's less copies of it that are stored and so if that data center and that availability goes down then you lose your object so of course it offered at the time the highest price reduction possible but now the difference between this one and the normal ia standard storage class is so small in terms of price that you're probably not going to migrate to or navigate to this storage class but it is still there in the documentation and it may very well come up still in the certification exam so at least be aware of that let's now take a look at some individual features of s3 starting off with life cycle management so life cycle management is very interesting because it allows us to come up with a predefined rule that will help us automate the transitioning of objects from one storage class to another without us having to manually copy things over of course you could imagine how time consuming that would be if we had to do this manually so we're going to see this very soon in a lab however let me discuss how how this works so once we it's basically a graphical user interface it's very very simple to use once you come up with these lifecycle management rules but you're going to define two things you're going to define the transition action and the expiration action so the transition action is going to be something like well i want to transition an object from maybe it's all objects or maybe it's just a specific type of object in a folder example that has a specific prefix from one storage class let's say standard to standard inactive or infrequent access maybe only after 45 days after at least a minimum of 30 days like we spoke of before and then maybe after 90 days you want to transition the objects in ia 2 right away glacier deep archive or 180 days you come up with whatever combination you see fit okay it doesn't have to be sequential from s3 to i8 to one zone et cetera et cetera because like we discussed before it depends what kind of objects that you're interested in putting in one zone objects that you don't really mind losing if that one availability zone goes down so you're going to be deciding those rules it ends up that this even is not a simple task because you have to monitor your usage patterns to see which data is hot which data is cold and what's the best kind of life cycle management to implement to reap the benefits of the lowest cost so you have to put somebody on this job and make the best informed decisions based on your access patterns and that is something that you need to consistently monitor so what we can do is we can instead opt for something called s3 intelligent tiering which basically analyzes your workload using machine learning algorithms and after about a good 30 days of analyzing your access patterns will automatically be able to transition your objects from s3 standard to s3 standard infrequent access okay it doesn't go past the ia one doesn't go after the glacier and whatnot okay so it can then offer you um that at a reduced um price overhead so there is a monitoring fee that is introduced in order to implement this feature it's a very nominal very very low monitoring fee and the nice thing is is if ever you take out an object out of the infrequent axis before the 30-day limit as we spoke of before you will not be charged an overhead charge because of that why because you're using the intelligence tearing you're already paying an overhead for the monitoring fee so at least in that sense the intelligent tearing will take the object out of ia and put it back into the s3 standard class if you need access to it before the 30 days and in that case you won't be charged at overhead so that is something that is very um that is very um good to to to do in order not to have to put somebody on that job so yes you're paying a little bit of overhead for that monitoring fee but at the other side of the spectrum you're not investing in somebody working many hours to monitor and put into place a system to monitor your data access patterns so let's take a look at how to do this right now let's implement our own lifecycle management rules so let's now create a lifecycle rule inside our bucket first off we're going to need to go to the management tab in the bucket that we just created and right on the top you see right away life cycle rule we're going to create lifecycle rule and we're going to name it so i'm just going to say something very simple like simply learn life cycle rule and we have the option of creating this rule for every single object in the bucket or we can limit the scope to a certain type of file perhaps with a prefix like i could see one right now something like logs so anything that we categorize as a log file will transition from one storage tier to the next as per our instructions we're doing this because we really want to save on costs right it's not so much of organizing what's your older data versus your newer data it's more about reducing that storage cost as your objects get less and less used so in this case logs are a good fit because perhaps you're using your logs for the first 30 days you're sifting through them you're trying to get insights on them but then you kind of move them out of the way because they become old data and you don't need them anymore so we're going to see how we can transition them to another pricing tier another storage tier we could also do this with object tags which is a very powerful feature and in the lifecycle rules action you have to at least pick one of these options now since we haven't enabled versioning yet what i'm going to do is just select transition the current version of the object between these storage classes so as a reminder of what we already covered in the slides our storage classes are right over here so the one that's missing is obviously the default standard storage class which all objects are placed in by default so what we're going to say is this we want our objects that are in the default standard storage class to go to the standard inactive access storage class after 30 days and that'll give us a nice discount on those objects being stored then we want to add another transition and let's say we want to transition them to glacier after 90 days and then as a big finale we want to go to glacier deep archive you can see the rest are grayed out would it make sense to go back and maybe after 180 days we want to go there okay now there's a little bit of a warning or call to attention here they're saying if you're going to store very small files into glacier not a great idea there's an overhead in terms of metadata that's added and also there's an additional cost associated with storing small files in glacier so we're just going to acknowledge that of course for the demonstration that's fine in real life you don't want to store very big tar files or zip files that had you know one or more log files in there okay that would bypass that that surcharge that you would get and over here you have the timeline summary of everything we selected up above so we have here after 30 days the standard inactive access after 90 days glacier and after 180 days glacier deep archive so let's go and create that rule all right so we see that the rule is already enabled and at any time you could go back and disable this if ever you had a reason to do so we can easily delete it as well or view the details and and edit as well so if we go back to our bucket now what i've done is created that prefix with the slash logs since we're not doing this from the command line we're going to create a logs folder over here that will fit that prefix so create logs create folder and now we're going to upload our let's say apache log files in here so we're going to upload one demonstration apache log file that i've created with just one line in there of course just for demonstration purposes we're going to upload that and now we have let's close that and now we have our apache log file in there so what's going to happen because we have that life cycle rule in place after 30 days anything any file that has the logs prefix or basically is placed inside this folder will be transitioned as per that life cycle road policy that we just created so congratulations you just created your first s3 lifecycle rule policy let's now move over to bucket policies so bucket policies are going to allow or deny access to not only the bucket itself but the objects within those buckets to either specific users or other services that are inside the aws network now these policies fall under the category of i am policy so im stands for identity and access management and this is a whole other topic that deals with security at large so there are no services in aws which are allowed to access other services or data for example within s3 without you explicitly allowing it through these im policies so one of the ways we do that is by attaching one of these policies which are written in a json format so it's a text file that we write at the end of the day that's the artifact and that's a good thing because we can use that artifact and we can configuration control it in our source control and version it and put it alongside our source code so we deploy everything it is part of our deployment package so in this case here we have several ways of doing this we can use what's called the policy generator which is a graphical user interface that allows us to simply click and point and populate certain text boxes which will then generate that json document that will allow us to attach that to our s3 bucket and that will determine like i said which users or services have access to whatever api actions are available for that resource so we might say we want certain users to be able just to list the contents of this bucket not necessarily be able to delete or upload new objects into that bucket so you can get very fine-grained permissions based on the kind of actions you want to allow on this resource so in order to really bring this home let's go and perform our very own lab on this let's now see how to create an s3 bucket policy going back to our bucket we're now going to go into permissions so the whole point of coming up with a bucket policy is that we want to control who or what the what being other services have access to our bucket and our objects within our bucket so there are several ways we can go about doing this let's edit a bucket policy one we can go and look at a whole bunch of pre-canned examples which is a good thing to do two we could actually go in here and code the json document ourselves which is much more difficult of course so what we're going to do is we're going to look at a policy generator which is really a form based graphical user interface that allows us to generate through the answers that we're going to give here the json document for us first question is we've got to select the policy type of course we're dealing with s3 so it makes sense for us to create an s3 bucket policy the two options available to us are allowing or denying access to our s3 bucket now in this case here we could get really fine grained and specify certain kinds of services or certain kinds of users but for the demonstration we're just going to select star which means anything or anybody can access this s3 bucket all right now depending on also the actions that we're going to allow so in this case here we can get very fine-grained and we have all these check boxes that we can check off to give access to certain kind of api action so we can say we want to give access to you know just deleting the bucket which obviously is something very powerful but you can get more fine grain as you can see you have more of the getters over here and you have more of the the the listing and the putting new objects in there as well so you can get very fine grain now for demonstration purposes we're going to say all action so this is a very broad and wide permission something that you really should think twice about before doing we're basically saying we want to allow everybody and anything any service all api actions on this s3 bucket so that's no small thing we need to specify the amazon resource name the arn of that bucket specifically so what we're going to do is go back to our bucket and you can see here the bucket arn okay so we're just going to copy this paste it in this policy generator and just say add statement you can see here kind of a resume of what we just did and we're going to say generate policy and this is where it creates for us and make this a little bit bigger for us it creates that json document so we're going to take this we're going to copy it and we're going to paste it into the generator okay now of course we could flip this and change this to a deny right which would basically say we don't want anybody to have access or any thing any other service to have access to this s3 bucket we could even say slash star to also encapsulate all the objects within that bucket so if i save this right now you have a very ironclad s3 bucket policy which basically denies all access to this bucket and the objects within of course this is on the other side of the spectrum very very secure so we might want to for example host a static website through our s3 bucket so in this case here allowing access would make more sense right so if i save changes you see that we get an error here saying that we don't have permissions to do this and the reason for that is because it realizes that this is extremely permissive so in order to give access to every single object within this bucket as in the case that i was stating of a static website being hosted on your s3 bucket it would be much better to also at first enable that option so i'm just going to duplicate the tab here and once you go back to the permissions tab one of the first things that shows up is this block public access setting right right now it's completely blocked and that's what's stopping us from saving our policy we would have to go in here unblock it and save it right and it's also kind of like a double clutch feature you have to confirm that just so you don't do that by accident right so now what you've effectively done is you've really opened up the floodgates to have public access to this bucket it's something that can't be accidentally done it's kind of like having to perform these two actions before the public access can be granted now historically this was something that aws was was guilty of was making it too easy to have public access so now we have this double clutch now that this is enabled or turned off we can now save our changes here successfully and you can see here that now it's publicly accessible which is a big red flag that perhaps this is not something that you're interested in doing now if you're hosting a public website and you want everybody just to have read access to every single object in your bucket yes this is fine however please make sure that you pay very close attention to this type of access flagged over here on the console so congratulations you just got introduced to your first bucket policy a permissive one but at least now you know how to go through that graphical user interface through the policy generator and create them and paste them inside your s3 bucket policy pane so let's continue on with data encryption so any data that you place in s3 bucket can be encrypted at rest very easily using an aes 256 encryption key so we can have server-side encryption we could have aws handle all the encryption for us and the decryption will also be handled by aws when we request our objects later on but we could also have client-side encryption where we the client that are uploading the object have to be responsible for also passing over our own generated key that will eventually be used by aws to then encrypt that object on the bucket side of course once that happens then the key is discarded the client key is discarded and you have to be very mindful that since you've decided to handle your own encryption client-side encryption that if ever you lose those keys well that data is not going to be recoverable in that bucket on the aws network so be very careful on that point we can also have a very useful feature called versioning which will allow you to have a history of all the changes of an object over time so versioning sounds exactly how it's named every time you make a modification to a file and upload that new version to s3 it will have a brand new version id associated with that so over time you get a sort of stack of a history of all the file changes over time so you can see here at the bottom you have an id with all these ones and then an id with one two one two one two so eventually if ever you wanted to revert back to a previous version you could do so by accessing one of those previous versions of course versioning is not an option that's enabled by default you have to go ahead and enable that yourself it is an extremely simple thing to do and so there may be a situation where you already have objects within your buckets and you only then enable versioning well versioning would only apply to the new objects that would get uploaded from the point that you enabled versioning the objects that were there before that point will not get a specific version number attached in fact they will have a sort of null marker the version number that will get attached to them it's only after that you modify those objects later on and upload a new version that they will get their own version numbers so right now what we're going to be doing is a lab on actual versioning so let's go ahead and do that right now in this lab we're going to see how to enable versioning in our buckets enable versioning is very easy we're simply going to click on our bucket go into properties and there is going to be a bucket versioning section i'm going to click on edit and enable it once that's done any new objects that are uploaded to that s3 bucket will now benefit from being tracked by a version number so if you upload objects with the same file name after that they'll each have a different version number so you'll have version tracking a history of the changes for that object let's actually go there and upload a new file i'll upload one called index.html so we're going to simulate a situation where we've decided we're going to use an s3 bucket as the source of our static website to deploy one and in this index.html file if you take a look right now let's take a look at what's in there you can see that we have welcome to my website and we're at version two okay so if i click on this file right now and i go to i can clearly see that there's some version activity that's happening here okay we have here um at 1456 which is the latest one the latest one is on the top the current version we have a specific version id and then we have a sort of history of what's going on here now i purposely enabled versioning before and then try to delete version or disable versioning but here's the thing with versioning you cannot disable it fully once it's enabled you can only suspend it right now suspending means that whatever version numbers those objects had before you decided to suspend it will remain so you can see i have an older version here that has an older version number and at this point here i decided to suspend versioning and so what it does instead of disabling the entire history it puts what's called a delete marker okay you could always roll back to that version if you want now in the demonstration when we started it together i enabled it again so you can see this is actually the brand new version number as we did it together but you don't lose the history of previous versioning if ever you had suspended before so that's something to keep in mind right and it'll come up in the exam where they'll ask can you actually disable versioning once it's enabled and the answer is no you can only suspend it and your history is still maintained now we have that version there and let's say i come to this file and i want to up grade this i don't know i say version three right and now what's going to happen is if i click on this version just as the current one with version two and i open this we should see version two which is fine that's that's that's expected if we go back to our bucket and upload that new version file that has version three in there don't just modify it we should now see in that index.html file a brand new version that was created under the versions tab and there you go 1458 just two minutes after you can see here we have a brand new version id right and if i open this one you can see version three so now you have a way to enable versioning very easily in your buckets and you also have seen what happens when you want to suspend versioning what happens to the history of those versions files before just to actually go back here to the properties where we enabled versioning the first place if i were to go back in here and disable it like i said you can't disable you can only suspend and that's where that delete marker gets placed but all your previous versions retained their version id so don't forget that because that will definitely be a question on your exam if you're interested in taking the certification exam so congratulations you just learned how to enable versioning let's move on to cross region replication or crr as it is known there will be many times when you find yourself with objects in a bucket and you want to share those objects with another bucket now that other bucket could be within the same account could be within another account within the same region or could be within a separate account in a different region so there's varying levels of degree there the good thing is is all of those combinations are available so crr if we're talking about cross-region replication is really about replicating objects across regions something that is not enabled by default because that will incur a replication charge because it's syncing objects across regions of course you are spanning a very wide area network in that case so there is a surcharge for that now doing so is quite simple to do but one of the things that we have to be mindful of is to give permissions for the source bucket which has the originals to allow for this copying of objects to the destination bucket so if we're doing this across regions of course we would have to come up with im policies and we would also have to exchange credentials in terms of im user credentials in terms of account ids and and such we're going to be doing a demonstration in the same account in the same region but largely this would be the same steps if we're going to go cross region so this is something you might find yourself doing if you want to share data with other entities in your company maybe you're multinational and you want to have all your lock files copied over to another bucket in another region for another team to analyze to extract business insights from or it might just be that you want to aggregate data in a separate data lake in an s3 bucket in another region or like i said it could be even in the same region or in the same account so it's all about organizing moving data around across objects across these boundaries and let's actually go through a demonstration and see how we can do crr let's now see how we can perform cross region replication we're going to take all the new objects that are going to be uploaded in the simply learn s3 demo bucket and we're going to replicate them into a destination bucket so what we're first gonna do is create a new bucket okay and we'll just tack on the number two here and this will be our destination bucket where all those objects will be replicated to we're gonna demonstrate this within the same account but it's the exact same steps when doing this across regions one of the requirements when performing cross region replication is to enable versioning so if you don't do this you can do it at a later time but it is necessary to enable it at some point in time before coming up with a cross region replication rule all right so let me create that bucket and now after the bucket is created i want to go to the source bucket and i want to configure under the managements tab here a replication rule so i'm going to create a replication rule call it simply learn rep rule and i'm going to enable this right off the bat the source bucket of course is the simply learn s3 demo we could apply this to all objects in the bucket or perform a filter once again let's keep it simple this time and apply to all objects in the bucket of course caveat here this will only now apply to any new objects that are uploaded into this source bucket and not the ones that are already pre-existing there okay now in terms of the a destination bucket we want to select the one we just created so we can choose a bucket in this account or if we really want to go cross region or another account in another region we could specify this and put in the account id and the bucket name and that other account so we're going to stay in the same account we're going to browse and select the newly created bucket and we're also going to need permissions for the source bucket to dump those objects into the destination bucket so we can either create the role ahead of time or we can ask this user interface to create a new role for us so we'll opt for that and we'll skip these additional features over here that we're not going to talk about in this demonstration i'm just going to save this so that will create our replication rule that is automatically enabled for us right now so let's take a look at the overview here you can see it's been enabled just to double check the destination bucket is the demo two we're talking about the same region [Music] and again here we could opt for additional um parameters like different storage classes in the destination bucket that that object is going to be deposited in etc etc for now we just created a simple rule now if we go back to the original source bucket which we're in right now and we upload a new file which will be transactions file in a csv format once this is uploaded that cross region replication rule will kick in and will eventually right it's not immediate but will eventually copy the file inside the demo 2 bucket now i know it's not there already so what i'm going to do is pause the video and come back in two minutes and when i click on this the file should be in there okay so let's now double check and make sure that object has been replicated and there it is been replicated as per our rule so congratulations you just learned how to perform your first same account s3 bucket region replication rule let's now take a look at transfer acceleration so transfer acceleration is all about giving your end users the best possible experience when they're accessing information in your bucket so you want to give them the lowest latency possible you can imagine if you were serving a website and you wanted people to have the lowest latency possible of course that's something that's very desirable so in terms of traversing long distances if you have your bucket that is in for example the u.s east one region in the united states in the virginia region and you had users let's say in london that want to access those objects of course they would have to traverse a longer distance than users that were based in the united states and so if you wanted to bring those objects closer to them in terms of latency then we could take advantage of what's called the amazon cloudfront delivery network the cdn network which extends the aws backbone by providing what's called edge location so edge locations are really data centers that are placed in major city centers where our end users mostly are located more densely populated areas and your objects will be cached in those locations so if we go back to the example of your end users being in london well they would be accessing a cached copy of those objects that were stored in the original bucket in the for example u.s east one region of course you will get most likely a dramatic performance increase by enabling transfer accelerations very simple to enable this just bear in mind that when you do so that you will incur a charge for using this feature best thing to do is to show you how to go ahead and do this so let's do that right now let's now take a look at how to enable transfer acceleration on our simply learn s3 demo bucket by simply going to the properties tab we can scroll down and look for a heading called transfer acceleration over here and very simply just enable it so what does this do this allows us to take advantage of what's called the content delivery network the cdn which extends the aws network backbone the cdn network is strategically placed into more densely populated areas for example major city centers and so if your end users are situated in these more densely populated areas they will reap the benefits of having transfer acceleration enabled because the latency that they will experience will be severely decreased so their performance is going to be enhanced if we take a look at the speed comparison page for transfer acceleration we can see that once the page is finished loading it's going to do a comparison it's going to perform first of all what's called a multi-part upload and it's going to see how fast that upload was done with or without transfer acceleration enabled now this is relative to where i am running this test so right now i'm actually running it from europe so you can see that i'm getting very very good results if i would enable transfer acceleration and my users were based in virginia so of course now i have varying differences in percentage as i go closer or further away from my region where my bucket or my browser is being is being referenced so you can see here united states i'm getting pretty good uh percentages as i go closer to europe it gets lower of course but still very very good frankfurt again this is about us probably at the worst i'm going to be getting here since i'm situated in europe and of course as i go look more towards you know the asian regions you can see once again it kind of scales up in terms of better performance so of course this is an optional feature once you enable it as i just showed over here this is a feature that you pay additionally for so bear that in mind make sure that you take a look at the pricing page in order to figure out how much this is going to cost you so that is it congratulations you just learned how to simply enable transfer acceleration to lower the latency from the end user's point of view we're now ready to wrap things up in our conclusion and go over at a very high level what we just spoke about so we talked about what s3 is which is a core service one of the original services published by aws in order for us to have unlimited object storage in a secure scalable and durable fashion we took a look at other aspects of s3 in terms of the benefits we mainly focused on the cost savings that we can attain in s3 by looking at different storage classes now of course s3 is industry recognized as one as the cheapest object storage services out there that has the most features available we saw what goes into the object storage in terms of creating first our buckets which are our containers high level containers in order for us to store our objects in again objects are really an abstraction of the type of data that are in there as well as the metadata associated with those objects we took a look at the different storage tiers the default being the standard all the way to the cheapest one which is the glacier which are meant for long term archived objects for example log files that you may hold on to for a couple of years may not need to access routinely and we'll have the cheapest pricing option by far so we have many pricing tiers and if you want to transition from one tier to the next you would implement a life cycle policy or use the intelligent tiering option that can do much of this for you we took a look at some very interesting features starting from the life cycle management policies that we just talked about all the way to versioning cross region replication and transfer acceleration so with this conclusion you are now ready to at least start working with s3 hello everybody my name is kent and today we're going to be covering aws identity and access management also known as i am tutorial by simply learn so these are the topics which we'll be covering today we'll be defining what aws security is the different types of security in aws what exactly is identity and access management the benefits of identity and access management how im works its components its features and at the end we are going to do a great demo on i am with multi-factor authentication and users and groups so without any further ado let's get started so let's get started with what is aws security your organization may span many regions or many accounts and you may encompass hundreds if not thousands of different types of resources that need to be secured and therefore you're going to need a way to secure all that sensitive data in a consistent way across all those accounts in your organization while meeting any compliancy and confidentiality standards that have to be met so for example if you're dealing with health care data or credit card information or personal identification information like addresses this is something that needs to be thought out across your organization so of course we have individual services in aws which we will explore in this video however in order to govern the whole process at an organizational level we have aws security hub now aws security hub is known as a cspm which stands for a cloud security posture management tool so what does that really mean well it's going to encompass like i said all these tools underneath the hood in order to bring a streamlined way for you to organize and adhere to these standards across organization it'll identify any misconfiguration issues and compliant risks by continuously monitoring your cloud infrastructure for the gaps in your security pulse reinforcements now why is that important well these misconfiguration can lead to unwanted data breaches and also data leakages so in order to govern this at a very high level like i said we need to incorporate aws security hub that will automate and manage all these underlying services for us in order to perhaps take automated action we call that remediary actions and you can approve these actions manually or you can automate those actions as you see fit across your organization so let's delve a little bit deeper into what is aws security and then we'll start looking individually at some of these services and so our organization has special needs across different projects and environments so of course the production development test environments all are going to have their own unique needs however it doesn't matter which project or which business unit we're talking about what are their storage backups needs they should all be implemented in a concise standardized weight across your organization to meet that compliancy we were just talking about so we want to automate all these tedious manual tasks that we've been doing like ensuring that perhaps our buckets and s3 are all encrypted or all our ebs volumes are encrypted and we want to have an automated process in place in order to give us time on other aspects of our business perhaps our application development to bring better business value and that allows to grow and innovate the company and a way that's best suited for it so let's take a look at the different types of aws security now there are many different types of security services out there however we're going to concentrate primarily on i am in this video tutorial so i like to think of i am as the glue between all aws services because by default we don't have permission for any one service to communicate with another so let's just take for example an ec2 instance that wants to retrieve an object from s3 well those two services could not interact unless we interacted with i am so something that's extremely important to learn and by the end of this video tutorial you'll be able to understand what im is we have amazon guard duty here which is all about logs basically aggregates all logs from example cloudtrail which we still haven't talked about at the end of this list over here but instead of looking at cloudtrail individually guard duty will actually take a look at trail cloudtrail we'll take a look at your vpc flow logs and we'll take a look at your dns logs and monitor that account and your network and your data access via machine learning algorithms and it'll id any threat where you can automatically remediate the workflow that you've approved so for example if you know of a malicious ip address that's making api calls well those api calls are registered in cloudtrail right so this machine learning algorithm will be able to detect that and take action so this is really governed at a higher level than you having to you know individually inspect all your dns locks flow logs and cloud trails individually and take action through some scripting fashion that that you've come up with so guard duty manages all that we have amazon macy which once again uses machine learning but also uses pattern matching pattern matching can be used with matching kind of libraries that are already in place or you can come up with your own pattern matching and that's used to discover and protect your sensitive data so for example if you had health care data also known as hipaa compliancy or credit card data that's lying around well macy will discover it and will protect it so as your data grows across organization that might be harder and harder to do in your own way so macy really facilitates uh discovering and protecting that so you can concentrate on other things aws config is something that kind of works in tandem with all the other services it's able to continuously monitor your resources configurations and ensure that they match your desired configuration so for example maybe you state that your s3 bucket should be encrypted by default all of them or you want to make sure that your im user access keys are rotated and if they're not then take remediary action so a lot of these automated remediary actions right are basically executed via aws config so you'll see the aws config actually used by other services underneath the hood and then you have cloudtrail cloudtrail is all about logging every single type of api call that's made so it'll it'll always record the api call who made it from what source ip the parameters that were sent with the api the response all that data which is really a gold mine for you to investigate if there's any security threat and again this trail over here and there can be many and there's lots and lots of data generated by these trails can be analyzed by these other services specifically guard duty here and that automates that process so you don't have to so let's get to the next one which is what is identity and access management so what is i am like i said im is the glue between all aws services it will allow those services to communicate with each other once you put im into place the other thing is is that allow us to manage our aws users which are known as im users and group them together in two groups to facilitate assigning and retrieving or removing permissions from them instead of doing it on an individual scale so of course if you had 200 im users and you wanted one shot to add a permission to a group that contained 200 item users well you can do that in one operation instead of 200. now we do have a distinction between i am users and end users and users use applications online or as im users are your employees in your organization that are interacting directly with aws resources so for example an ec2 instance would be a resource that let's say a developer would be interacting with and traditionally those are resources that are used full time from nine to five let's say five days a week 365 days a year okay now sometimes those users have to have elevated permissions for a temporary amount of time and that is more suited for a role and that will eliminate the need to create separate accounts just for you know type of actions that are needed for let's say an hour a month or two hours a week or something like that something like backing up an ec2 instance removing some files doing some cleanups traditionally you don't have those permissions as a developer but you can be given temporary permissions to assume this role so it's kind of like putting a hat on your head and pretending you're somebody that you're usually not for a temporary amount of time once you take off the hat you go back to being your old boring imuser self which doesn't have access to performing backups or cleanups and stuff like that so the roles interact with a service called a secure token service which gives us specific keys threes specifically an access key a secret key much like a username and password and then we have a token key which only gives you access to this role this elevated permission for 1 to 12 hours so we'll talk more about roles as we go on through this video tutorial but it's important that you at least remember at this point even if you don't know all the details that roles have to do with temporary permissions elevated permissions so what are the benefits of im well across our organization we're going to have many accounts and like i said hundreds if not thousands of resources so scalability is going to be an issue if we don't have a very high level uh visibility and control over the entire security process and once we have a tool like for example security hub which we saw on the first slide we can continuously monitor and maintain a standard across our organization so very very important to have that visibility and we do when we integrate with aws security now we also need to eliminate the human factor right so um we don't want to manually put out fires every single time of course there will be times when something new occurs that we've never seen before that that you know will be needed however once an occurrence happens and reoccurs we can obviously come up with a plan to remediate the action so once we automate we can definitely reduce the time to fix that reoccurring error or it could be a new error that we don't have to interact with because we're using machine learning algorithm services like i was talking about like our duty or macy and we can really reduce the risk of security intrusions and data leakage and such by using im to facilitate this of course you may have many compliance needs you may be dealing with applications that use health care data or credit card information for payments or you might be dealing with a government agency let's say in the u.s that has certain compliancy requirements that needs assurances that if their data is stored on the cloud that you're still following the same compliance controls that you were let's say on premise so we have a very very long list of compliance requirements that each aws service adheres to and you can go on the um aws documentation online and you could figure out if for example dynamodb is compliant with hipaa compliancy and it is and so you can be sure that you can use that so aws itself has to constantly maintain these compliancy controls in place and they themselves have to pass all these certifications and that gets passed on to us so much less work for us to implement these compliance controls we can just inherit them by using aws a security model by im and last but not least we can build the highest standards for privacy and data security now having all this on our own on-premise data center poses security challenges especially physical security challenges you have to physically secure the data center you might have security personnel that you need to hire 24 7 camera surveillance etc etc so just by migrating to the cloud we can take advantage of aws's global infrastructure they're very very good at building data centers that's part of their business and securing them so of course we have a shared security model in place however you can rest assured that at least as the lowest common denominator physical security has been put into place for us again as other services are used more managed services or more higher-end services used in aws more and more of that security model will be aws responsibility and less yours we will always have a certain type of responsibility for our applications but we can again use these high-level services to ensure that our ability to encrypt the data at rest and in transit are always maintained by coming up with specific rules that need to be adhered to and if those rules are not adhered to then we have remediary actions that take place in order to maintain again that cross account or organizational wide security control okay so lots of benefits of using i am and let's now take a look at exactly how im works and delve deeper into authentication and authorization so there are certain terms here that we need to know about one of them is principle now principle is nothing more than an entity that is trying to interact with an aws service for example an ec2 instance or an s3 bucket now that could be a user an im user it could be even a service or it could be a role so we're going to take a look how principles need to be specified inside i am policies authentication so authentication is exactly how it sounds who are you could be something as simple as username and password or it could involve an email address example if you're the root user of the account when you're creating the account for the very first time other normal users will not need to log in with their user email it'll only be the root user that needs that you could also have developer type access which needs access let's say via command line interface or a software development kit also known as an sdk now for those kind of access points we're going to need access keys and secret keys or public private keys let's say if you want to even authenticate and log in through ssh to an ec2 instance so there are many different types of authentication that we could set up when creating i end users based on what kind of access they need do they just need access to the console do they need access to just services uh in terms of as a programmatic needs so those are different types of authentication and then we have the actual request now we could make a request in aws through various ways we're going to be exploring that via the console the aws console however every button you click every drop down list you select that invokes an api call behind the scenes so everything goes behind a centralized well documented api every service has an api so if you're dealing with lambda let's say well lambda has an api if you're dealing with ec2 ec2 has an api now you could interact with that api like i said via the console indirectly you can use a command line interface you can use an sdk for your favorite programming language like python so all of them get funneled through all your requests get funneled through an api but that doesn't mean you're allowed to do everything just because we know who you are and you have access to some keys let's say to make some api calls you have to be authorized in order to perform certain actions so maybe you're only authorized to communicate with dynamodb and also maybe you're only authorized to perform reads and not right so we can get some very fine-grained authorization through some in policies in order to control not only who has access to our aw resist but what they're allowed to do so very fine grained actions read actions right actions both maybe just describe or list some buckets so depending on the aws user that's logged in we can control exactly what actions because every api has fine grained actions that we can control through im and of course many many different types of resource when we're coming up with these policies all these actions can be grouped into resources so perhaps we can say well this user only has access or read only access to s3 specifically these buckets and can write to a dynamodb table but cannot terminate or shut down an ec2 instance so all these kind of actions can be grouped into resources on a pro-resource basis based on the kind of role you have in the projects or in the company so now let's take a look at the components of im we've already seen that we can create i am users and that how they are different than your normal end users now those entities represent a person that's working for organization that can interact with aws services and some of those permissions that we assign through what's called identity based policies to such a user will have permissions for example that are always necessary every time that they're logged in so perhaps this user tom always needs access to the following services over here for example an rds instance or an s3 bucket etc etc and sometimes they will need temporary access to other services for example dynamodb so in those cases there would be a combination of normal im policies assigned to the user and certain assume role calls done at runtime in order to acquire this temporary credential elevated permission there is also the root user which is different than your typical super user administrator access there are some things that a root user can do that administrator access cannot for example a root user first of all logs in with their email address and can also have the option to change that email address something that is very tightly coupled to the aws account so you cannot change the email address with an administrator account the other thing your user can do is they can change the support contract they can also view billing information or taxes information that information for example so the best thing to do the best practice is that once you've created your aws account you are obviously the root user the best thing to do is to enable multi-factor authentication store away those keys in a secure location and your first administrative task should be to create an administrator super user from which that point on you will log in as only an administrative user and you will create other type of administrative users and those administrative users will create other i am users so never log into your aws account as a root user unless you need to access that specific functionality that i said that is over and above a super user administrator account so here we have a set of im users and we could assign to them directly im policies which give them access to certain services however the best practice is to create groups which is just a collection of im users and suppose we have a developer group and a group of security administrators of course we could assign different types of policies to that group and every user that is assigned to the developer group will inherit those im permissions makes it much easier to manage your im users this way for and for the fact that also you can have users that can be assigned to groups as you can see here but what's not shown here that is possible as well is you could have a user that is part of two groups at the same time the thing that you cannot have though is the notion of sub group so i can't extend this group and create a smaller group of developers out of this big group here so we don't have hierarchical groups but we do allow users to partake in different groups at the same time so they will inherit those permissions those iem policy permissions from for example if user a was part of developers and user a was part of security they would inherit these two policy uh statements over here now there is another type of iam policy and that is called a resource-based policy so resource-based policies are not attached directly to for example users but they're attached more to resources like an s3 bucket for example and so when you have such a case a user group can't be designated as a principle to a group that's just one of the restrictions however there are other ways to get around that as i said when you think of an i am role i want you to think that these are temporary credentials that are required at runtime and are only good for a specific amount of time now an iem roll is really made up of two components we have what's called a trust policy so who do you trust to assume this role who do you would trust to even make an api call to assume the role after that we have an iron policy once you've been trusted to assume the role and you get those temporary tokens what are you allowed to do maybe you're allowed to have read write access to the dynamodb so here we have no long-term credentials that are connected to role you do not assign a role to a user the user or service will assume the rova api call via the secure token service sts and with those temporary credentials we'll then be able to access for example an ec2 instance if the im policy attached to the item role says that you can do so so your users and your applications perhaps running on an ec2 instance and the awf aws services themselves don't normally have access to the aws resources but through assuming a role they can dynamically attain those permissions at runtime so it's a very flexible model that allow you to bypass the need to embed security credentials all over the place and then have to maintain those credentials and make sure they're rotated and they're not acquired by anybody trying to find a security hole in your system so from a maintenance point of view they're really an excellent tool to use in your toolbox your security toolbox so let's take a look at the components of i am we have the i am policy over here which is really just a json document and that json document will describe what this user can do if it's attached directly to it or if the iron policy is attached to the group whatever user is attached to that group will also have the same policy inherited so these are more long-term credentials in terms of or permissions rather that are attached to your login for the duration of your login whereas roles are as we've already stated temporary and again once you've established a trust relationship then we can assign a policy for example a policy that gives you only read only access to a cloud trail to a role now it's up to the user to assume that role either through the console or through some programmatic means whichever way is performed the user will have to make a direct or indirect call to sts for the assume roll call now these permissions are examined by aws when a user submits a request so this all happens at runtime dynamically so when you change a policy it will take effect pretty much immediately as soon as the user makes the next request that new security policy whether you've just added or removed the permission will take immediate effect so what we're going to do now is just really quickly go to the aws console and actually show you how an im policy looks like so i've just logged into my aws management console and if i search for identity and access management it should be the first one to show up i'll just click on that and i want to show you actually how these i am policies look like if you go to the left hand side here there's an entry for policies and there are different types of policies that we can filter out so let's filter them out by type we have customer managed we have aws managed and more in line with job functions that aws managed also so we're going to take a look at aws manage all that means is that aws has created and will actually update maintain these pre-established and already vetted identity policies i am identity policy so i could further down maybe look at all the s3 policies that are available and we can see there are some that allows only read only access there are some that will give us full access and there are of course other types here that we're not all going to explore but let's just easily kind of take a look here at the full access you can see it's just a json document and of course you cannot modify it since this is a managed policy however you can copy it and modify it in that case it will become a a your own version of it and so it'll be a customer defined and so when you go here and you do a filter you can filter on your own customer managed policies so over here we are simply allowing certain actions right we talked about actions before that were based on api so here we have an api that starts with s3 so this is for the s3 api and instead of filtering out exactly specifically what api calls are there we're basically saying all api calls so star representing all api actions and again there is another category of s3 which deals with object lambda and we are allowing all operations on that specific api as well we could if this was our own customer managed ident policy actually scale down on which resource which bucket let's say or which folder in a bucket this would apply to but in this case since this is a full access we're basically being given permission to execute every single api action under s3 for all buckets we're going to see in the lab where the demonstration at the end of this tutorial that i'm going to show you how to scale that down okay so that's how an im policy looks like and if you want to assign an im policy to a user well stay tuned for the demonstration at the end of the video tutorial what's not at the end of video tutorial is really how to create a role so i'll show you what a role is here again there's a whole bunch of roles already uh set up that we can go and and take a look at here but we can go and create our own role as well and i might create a role for example that gives an ec2 let's choose choose ec2 access to s3 so if i click ec2 and click next permissions i can actually assign the s3 permission right over here just a policy just like we took a look at obviously assign it a tag and review it and i'm just going to call this my ec2 role 2s3 create the role and because i selected ec2 as a service it automatically will include that in the trust policy so now when i go and i create an ec2 instance i can attach this row to that ec2 instance and that ec2 instance will automatically be able to if we're using the linux 2mi let's say have access to or full admin access to any s3 bucket in the aws account so let's go take a look at where we would actually attach that role on the ec2 so when you're creating an ec2 let's just say we go here we say launch an ec2 instance and we'll pick this ami over here because it'll already include the packages necessary to perform the assume rule call to sts4a so we won't have to code that and we don't have to embed any credential keys in order to get this running which is really good which is the whole point of what i'm doing because by assuming a role i don't have to manage that and i'm just going to pick the free t2 micro and it is here once you you know pick whatever you need whatever vpc and whatnot then you come to the i am roll and you select the roll to be assumed dynamically now i have a whole bunch of them here but this is the one that i had created here my ec2 roll to s3 and if i go and i launched this ec2 instance and then i had an application let's say that i needed to contact s3 or even if i went on the command line of the ec2 instance i would be able now to communicate with the s3 service because they would use they being aws would actually use this role to perform an assume roll call to get those temporary sts tokens for me which will then allow me to access my im policy which gives me full access to s3 and off i go so this is a great way like i said to avoid having to administrate any embedded keys in this ec2 instance and i could take this role away at any time as well so there you have it a little demonstration on how to create a role how to attach it to an ec2 instance and also how manage and customer managed policies look like in i am of course there is also the fact that we have multiple accounts and so we could assign a policy that will also allow a user in another account access to our account so cross account access is something that's very very well needed and also is a feature of i am so when you need to share access to resources from one account to another in your organization we are in need of an i am policy to facilitate that so also roles can come into play here for that by also assigning granular permissions we can make sure that we implement the concept or the best practice of the least privileged access and also we have secure access to aws resources because by default again we have the principle of least privilege and no service can communicate with another until we enable it via im so very very secure out of the box of course if you want to add an additional layer of authentication of security we have multer multi-factor authentication rather and that will allow you to ensure through a hardware device perhaps a let's say a hardware token device or your cell phone that has some software that generates a code that will allow to identify you are actually the person that is typing the username and password and that's somebody who's actually stolen your username and password so at the end of this video tutorial we will show you or i will show you how to enable that and of course identity federation is a big thing where your users may have been defined outside of aws you may have thousands of users that you've already defined let's say in an active directory or in an ldap environment on-prem and you don't want to recreate all those users again in aws as i am user so we can tie in we could link the users your user database that's on-prem or even users that have been your user account that's been defined let's say uh on a social site like um facebook or twitter or google and tie that into your aws account and though there is an exchange of information there is a setup to do but at the end of the day it is either a role in combination with an i am a policy that will allow to map what you're allowed to do once you've been authenticated by an external system so the this has to do with perhaps like i said ldap or active directory and you can tie that in with aws sso there are many different services here that can be leveraged to facilitate what we call identity federation and of course im itself is free so it's not a service per api request aws is very concerned to put security first and so i am is free it's just the services that you use for example if you're securing an ec2 of course that ec2 instance falls under the current pricing plan so of course compliance like i mentioned is extremely important making sure that it's business as usual in the cloud and so for example if you were using the payment card industry data security standard to make sure that you're storing and processing and transmitting credit card information appropriately in order to do your business well you can be assured that the services you use in aws are pci dss compliance and there's many many other types of compliance that aws adheres to as well so password policies of course are important by default we do have a password policy in place but when you go to the im console you are free to update that password policy make it more restrictive based on whatever policy you have at your company so many features and so i think we're ready now to go into a full demonstration and i will show you how to incorporate i am users groups and also multi-factor authentication so get ready let's do it i'm going to demonstrate now how to attach an s3 bucket policy via im i've already created a bucket called simply learn s3im demo and what we're going to do is we're going to attach a bucket policy right over here we're going to have to edit it of course and the basis of the demonstration will be to allow a user that has mfa so multi-factor authentication set up to have access to the contents of a folder within this bucket so of course now i have to create a folder just simply call it folder one create that folder and now i'm going to go into that folder and upload a file so that we can actually see this in action so i'm going to select a demo file that i prepared in advance called demo file user 2 mfa i'm going to upload that all right and now what's going to happen is eventually when i create our two im users one will have access to view the contents of this file which should look like something very simple if i open this up within the console you'll be able to see well it's pretty small right now i'll make this a little bit bigger of course it says this is a demo file which user 2-mfa should only be able to have access to so what's left to do now is to create two users one called user one and one called user two dash mfa and that's what we're gonna do right now let's head over to i am identity and access management and on the left hand side here we're going to click on users you can see here i don't have much going on i just got an administrator user so i want to create two new users we'll start with one at a time so one called user one now by default we have um you know the principle of least privilege which means this user is not allowed to do anything unless we give them access to either the console right over here which i will do right now and also if they were a kind of programmatic user that needed access to specific keys in order to interact with the apis via the command line interface or the software development kit for example if you were a java developer so in this case that is not what i want so i'm just going to assign console access and i'm also going to create a default password for this user but i will not allow them to reset it on the first login just for demonstration purposes now over here we have the chance to add the user straight to a group but i'll only do that later on at the end of the demonstration for now i want to show you how to attach to this user and i am policy which will give them access to specific services so if i type in s3 full access here this is an administrative access to an s3 bucket if you take a look at the json document attached or representing that i am policy you can see that it is allowing all api actions across s3 star and also object lambda across all buckets and i say all buckets because the star represents all buckets so it is a very broad permission so be very careful when you're actually assigning this to one of your users to make sure that they deserve to have such a permission so i'm not going to assign any tags here but i'm just going to review what i've done i can see here i've just attached an s3 full access permission to this one user and i'm going to create that user now of course now is the time for me to download the csv file which will contain those credentials if i don't do it now i will not have a second chance to do this and also to send an email to this user in order for them to have a link to the console and also some instructions i'm not going to bother with that i'm just going to say close and now i have my user 1. you can see here that it says none for multi-factor authentication so i have not set anything up right now for them and that's what i want to do for this second user that i want to create called user 2 vash mfa or mfa rather all right so now i want to do the same exact thing as before i'm going to assign a custom password and i'm going to go through the same steps as we saw before i'm going to attach an existing managed policy that aws has already vetted for us review create now right now they're exactly the same so i have to go back to user two and set up multi-factor authentication and to do that you have to go in the security credentials tab and you'll see here assigned mfa device we have not assigned any yet so we're going to manage that right now and we have the option to order and get an actual physical device that's uniquely made for performing multi-factor authentication however we're going to opt for the virtual mfa device which is a software that we're going to install on our cell phones let's say so i can actually open this link just to show you the um very many options that we have for that right over here i'm gonna go with google authenticator so go to the app store on your android device and you can download that of course if you have an android device if not you have other options for iphone and other softwares for each one of those so i'll let you do that once you've installed that software on your cell phone you could come here and show the qr code now what that is going to do is you're going to need to go to your phone now and i'm going to my phone right now as we speak and i'm going to open up that mfa software in this case the google authenticator and i'm going to say there's a little plus sign at the bottom i'm going to say scan qr code i'm going to point my phone to this square code here this qr code and it's going to pick it up i'm going to say add account there's a little button there out of count and it's going to give me a code so i'm going to enter that code right now so once i've entered the code i have to wait for a second code and that's because this code on the screen of my cell phone is going to expire it's only there for about 20 to 30 seconds once that token expires it refreshes the display with another token so i have another token now that's only good for a couple of seconds again and now i'm going to say assign mfa what this does is it completes the link between my personal cell phone and this user right now so now that this has been set up we can see we actually have an arn to this device which we are going to possibly need depending on how we write our policies in our case here i'm going to show you a way to kind of bypass having to put that so now we've got those two users one with an mfa device and another one without okay so now the lab is going to be or the demonstration rather is going to be how do we allow user to mfa access to this file in folder one and not allow user one the distinguishing factor will be that user two has an mfa set up and user one doesn't so based off of just that condition that'll be the determining factor if they can view the file or not so let's get back to s3 and actually create this i am bucket policy and see how it's done so back we are in our bucket and now we have to go to permissions in order to set up this bucket policy we're going to click on edit over here and we're going to click on policy generator to help us generate that json file through a user interface of course we're dealing with an s3 bucket policy now what we're going to do is we're going to deny all users and services so star s3 is great out here because we selected s3 up here we want to deny all s3 api actions we're not going to go and selectively go down the list the specific bucket and the folder so in this case here we're missing the amazon resource name so we have to go back here and we have to copy the arn of our bucket right so we're going to copy that come back ctrl v and we're going to of course have to specify the folder as well so i'm going to just do slash folder one and then i'm also going to do slash star for all the objects in that folder okay so here's the mfa condition that i want to add though because we cannot assign the mfa condition up here we have to say add conditions very powerful feature here that's optional but very powerful we want to look for a condition that says boolean if exist or bool if exists all right so there's got to be a key value pair that exists and there are many but we're going to look for one that is specifically known as multi-factor authentication present and we want to make sure that value is equal to false so i'm going to add that condition and then i'm going to add that statement and i want to generate the policy just to show you now how that user interface has generated this json document for us that we're going to attach to our s3 bucket what i want to do is i want to take this in its entirety copy it and go back to our s3 bucket that's just waiting for us to paste our policy in here okay so let's see what's actually going on here we've created a policy again this policy could have any id it's just a an id that was given by the user interface you could name that whatever you want of course something that makes total sense and then we have one or more statements and then we only have actually one statement and again you can select any id here that was just auto-generated by the ide we're saying that all actions on the simple storage service i say all again because of the star are denied specifically on this resource which is our bucket and the folder named folder1 and all the objects within that folder under the certain conditions so the condition is if you have a boolean value of false associated with a key called multi-factor authentication present and it has to exist okay then if that value is equal to false you don't have access to the objects in this folder which means that if you do have mfa set up you will be allowed access to the contents of this folder so if we assign this and apply this to all principles which would encompass user 1 and user 2-mfa then let me just apply this that would mean that user one would not be able to see the contents of our object inside folder one and user two dash mfa would so let's go take a look and log into let's log out first and then when we're going to log back in we're going to log in as the individual user1 and user2mfa and prove this is actually the case so i am now logging back in as user one that we created moments ago and of course if i for example go see c service like ec2 we don't have access to ec2 we've just given ourselves access to s3 so this makes total sense we're just going to go see what we have access to which is s3 should be able to list and describe all the buckets there because we've been giving pretty much administrative access now if we go here to our created bucket and go inside folder 1 and want to actually open up this document we can see now that that's not happening okay and that's because of that condition that is checking if we actually have a value of true for that um mfa key that we specified okay so this is expected and this is exactly what we wanted now what we're going to do is we're going to log out once again and we're going to do the same thing but with user 2 dash mfa let me log back in and then specify the new password we created now of course now i have an mfa code right because i've set that up for user two so back on my cell phone i go i am opening up the google authenticator application and i have in front of me a new amazon web services code that is only good for like i said maybe 20 seconds so it's a little nerve racking if you're if you're not good under pressure you're gonna choke so i've just entered the code and i have to wait for uh you know that to actually work now it looks like i actually didn't check off um not to recreate the the password so unfortunately uh i must have missed that check mark so i wanted to kind of avoid this here so i'm gonna start over and create a brand new password a little painful for you know demonstrations but it's good for you guys to see how that actually looks like so because i've set up my mfa device properly i am able to log in after i enter that mfa code and once again if i go to ec2 now i haven't been given the permissions for ec2 i've only been given the permissions for s3 so nothing changes here i'm going to go to s3 and i'm going to go into folder one of our bucket in question and i'm going to see if i have access now to opening up and seeing the contents and here i have the contents this is a demo file which user2mfa should only be able to have access to so that worked so now you guys know how to create a bucket permission policy that is also known as a resource policy because it's assigned to the resources assigned to the bucket and let me actually show it to you once one last time here it is and that'll only allow us to have access to the contents of folder one if we have multi-factor authentication set up in our account so that's really good the last thing i want to show you now is how to maybe create im users in a better way in terms of not having to assign over and over again the same permissions like you saw me do for user one and user two of course now i cannot do anything i am why because i'm user i'm user two i don't have access to im so once again i'm logging out and we're going to actually see the best practice in terms of how do i assign i am policies to a group and then assign users to that group in order to inherit those permission policies so let's get to it let's sign up first so let us go back to i am and we're going to be creating our first group we're going to simply create the button or click on the button create group and we'll come up with a group called testers we have the ability to add existing users to this group so we will add for example user 1 only to the testers group and we can also add permissions to this group so we're going to go here and add for example the ec2 [Music] full permissions create group and what's going to happen now if we go take a look at testers we can see we have that one user that we added and we also have the permissions associated with that group so this means now that any new users we create will automatically inherit whatever permissions the group has which is really good for maintainability so for example let's say we were to create many new testers so let's say user three over here now of course i'm going to have to go in this case through the same steps as i did before i'm just going to remember this time not to forget to check uncheck that and instead of attaching policies directly i'm going to say well i want to inherit the policy permissions that were assigned to the group so i'm going to say this user is going to be part of the testers group and of course i'm just going to skip through the remaining repetitive tasks that i've already described and this user 3 now is already going to have their permissions to access ec2 that they've gained through the group testers whereas if we go back to user one this user also has the permissions from the group but has a unique permission added to itself so in this case you do have the option of doing that you have the option of adding a permission over and above that is assigned to the group to yourself directly so this is really good adding permissions to groups when you have of course lots of users in your company in your organization and instead of going to them and adding permissions after permission you can centrally access this in one or manage this in one place so for example if i go back to my roles groups rather in my testers and let's say i forgot to add a permission to all the users in this group so let's say there was 20 users attached to this group okay i'm not going to bore you with creating 20 users and we have we have two now that's good enough to show the demonstration and let's say now i need to add another permission right so in this case here i'm going to add a permission for example let's say based on dynamodb right so i'm going to get full access to dynamodb i'm going to add that permission and what's going to happen now is if i go back to my users for example user 3 i'm going to see that user 3 automatically has inherited that permission and so has user one all right show two more and there they are so in this case it really allows for you to easily manage a lot of users in that group because you can simply go to one place which is the group and add and remove policies as you see fit and the whole team will inherit that so it saves you lots of work and that is the best practice so there you have it guys you have learned in this demonstration how to create users how to create groups how to add users to groups and manage their policies in terms of best practice and you've also learned how to attach to an s3 bucket a policy that allows you to permit access based off of if a user has multi-factor authentication set up and we showed you how to actually set that up when creating an im user so again permissions tab is where this all happens hope you enjoyed that put that to good use and i'll see you in the next demonstration so i hope you enjoyed that demonstration we're just going to wrap things up in a summary and kind of see what was looked at throughout the tutorial we first started with what is aws security we took a look at how important it was across our organization to maintain best practice and also standardize that practice across accounts organization we took a look at the topmost services in which we concentrated then after on i am and we took a look at what exactly what i am was in terms of i am users groups long-term credentials that are applied as i am policies and then the concepts of a role we took a look at the benefits of im and the actual terminology used with working with i am of course principle being an entity that is a either a user or a service itself that can gain access to an aws resources and we actually saw the different types of authentication and what we can do to implement authorization via im policies and also through the use of roles so we took a look how to organize our users into groups and what actually goes into acquiring a role through a demonstration we took a look at how to actually create a role and attach that role to an ec2 instance and then we took a look at the high level features of im which allowed us to either grant access to another imuser from another group implement multi-factor authentication like we just did in the demonstration or ensure ourselves that we are following all the compliancy standards that have been adhered to by whatever project we're working on and that aws is there to support us so here we're going to talk about amazon ecs a service that's used to manage docker containers so without any further ado let's get started in this session we would like to talk about some basics about aws and then we're going to immediately dive into why amazon ecs and what is amazon ecs in general and then it uses a service called docker so we're going to understand what docker is and there are competitive services available for ecs i mean you could ecs is not the on and only service to manage docker containers but why ecs advantage of ecs we will talk about that and the architecture of ecs so how it functions what are the components present in it and what are the functions that it does i mean each and every component what are all the functions that it does all those things will be discussed in the architecture of amazon acs and how it works how it all connects together that's something we will discuss and what are the companies that are using ecs what were the challenge and how ecs helped to fix the challenge that's something we will discuss and finally we have a wonderful lab that talks about how to deploy docker containers on an amazon ecs so let's talk about what is aws amazon web service in short called as aws is an web service in the cloud that provides a variety of services such as compute power database storage content delivery and a lot of other resources so you can scale your business and grow not focus more on your id needs and the rest of the id demands rather you can focus on your business and let amazon scale your it or let amazon take care of your it so what is that you can do with aws with aws we can create deploy any application in the cloud so it's not just deploying you can also create your application in the cloud it has all the tools and services required the tools and services that you would have installed in your laptop or you would have installed in your on-premises desktop machine for your development environment you know the same thing can be installed and used from the cloud so you can use cloud for creating and not only that you can use the same cloud for deploying and making your application available for your end user the end user could be internal internal users the end user could be could be in the internet the end user could be kind of spread all around the world it doesn't matter so it can be used uh to create and deploy your applications in the cloud and like you might have guessed now it provides service over the internet that's how your users worldwide would be able to use the service that you create and deploy right so it provides service over the internet so that's for the end customer and how will you access those services that's again through the internet it's like the extension of your data center in the internet so it provides all the services in the internet it provides compute service through the internet so in other words you access them through the internet it provides database service through the internet over the internet in other words you can securely access your database through the internet and lot more and the best part is this is pay as you go or pay only for what you use there is no long term or you know beforehand commitment here most of the services does not have any commitment so there is no long-term and beforehand commitment you only pay exactly for what you use there's no overage there's no overpaying right there's no buying in advance right you only pay for what you use let's talk about what ecs is so before ecs before containers right ecs is a service that manages docker containers right it's not a product or it's not a feature all by itself it's a service that's dependent on docker container so before docker containers all the applications were running on vm or on a host or on a physical machine right and that's memory bound that's latency bound the server might have issues on and on right so let's say this is alice and she's trying to access her application which is running somewhere in her on premises and the application isn't working what could be the reason some of the reasons could be memory full the server is currently down at the moment we don't have another physical server to launch the application a lot of other reasons so a lot of reasons why the application wouldn't be working in on premises some of them are memory full issue and server down issue very less high availability or in fact single point of failure and no high availability if i if i need to tell it correctly with easiest the services can kind of breed free right the services can run seamlessly now how how is that possible those thing we will discuss in the upcoming sessions so because of containers and ecs managing containers the applications can run in a high available mode they can run in a high available mode meaning if something goes wrong right there's another container that gets spun up and your application runs in that particular container very less chances of your application going down that's what i mean this is not possible with a physical host this is very less possible with an vm or at least it's going to take some time for another vm to get spun up so why ec is or what is ecs amazon ecs maintains the availability of the application and allows every user to scale containers when necessary so it not only meets the availability of the application meaning one container running your application or one container hosting your application should be running all the time so to meet that high availability availability is making sure your service is running 24 7. so container makes sure that your services run 24 bar 7 not only that not only that suddenly there is an increase in demand how do you meet that demand right let's say you have like thousand users suddenly the next week there are like two thousand users all right so how do you meet that demand container makes it very easy for you to meet that demand in case of vm or in case of physical host you literally will have to go buy another physical host or add more ram add more memory add more cpu power to it all right or kind of club 2 two three hosts together clustering you would be doing a lot of other things to meet that high availability and also to meet that demand but in case of uh ecs it automatically scales the number of containers it automatically scales the number of containers needed and it meets your demand for that particular r so what is amazon ecs the full form of ecs is elastic container service right so it's basically a container management service which can quickly launch and exit and manage docker containers on a cluster so it's the function of ecs it helps us to quickly launch and quickly exit and manage docker container so it's kind of a management service for the docker containers you will be running in amazon or running in the aws environment so in addition to that it helps to schedule the placement of container across your cluster so it's like this you have two physical hosts you know joined together as a cluster and ecs helps us to place your containers now where should your container be placed should it be placed in host 1 should be placed in host 2. so that logic is defined in ecs we can define it you can also let ecs take control and define that logic most cases you will be defining it so schedule the placement of containers across your cluster let's say two containers want to interact heavily you really don't want to place them in two different hosts right you would want to place them in one single host so they can interact with each other so that logic is defined by us and these container services you can launch containers using aws management console and also you can launch containers using sdk kits available from amazon you can launch through a java program you can launch container using and.net program you can launch container using an node.js program as in when the situation demands so there are multiple ways you can launch containers through management console and also programmatically and ecs also helps to migrate application to the cloud without changing the code so anytime you think of migration the first thing that comes to your mind is that how will that environment be based on that i'll have to alter my code what's what's the ip what is the storage that's being used what what are the different parameters i'll have to include the environment parameters of the new environment with containers now that worry is already taken away because we can create and pretty exact environment that the one that you had an on-premises the same environment gets created in the cloud so no worries about changing the application parameter no worries about changing the code in the application right you can be like if it ran in my laptop a container that i was running in my laptop it's definitely going to run in the cloud as well because i'm going to use the same container in the laptop and also in the cloud in fact you're going to ship it you're going to move the container from your laptop to amazon ecs and make it run there so it's like the same the very same image the very same container that was running in your laptop will be running in the cloud or production environment so what is docker we know that it ecs helps to quickly launch exit and manage docker containers what is docker let's let's answer that question what is docker now docker is a tool that helps to automate the development of an application as a lightweight container so that the application can work efficiently in different environments this is pretty much what we discussed right before the slide i can build an application in my laptop or in on premises in a container environment docker container environment and anytime i want to migrate right i don't have to kind of rewrite the code and then rerun the code in that new environment i can simply create an image docker image and move that image to that production or the new cloud environment and simply launch it there right so no compiling again no relaunching the application simply pack all your code in a docker container image and ship it to the new environment and launch the container there that's all so docker container is a light weight package of software that contains all the dependencies so because you know when packing you'll be packing all the dependencies you'll be packing the code you'll be packing the framework you'll be packing the libraries that are required to run the application so in the new environment you can be pretty sure you can be guaranteed that it's going to run because it's the very same code it's the very same framework it's the very same libraries that you have shipped right there's nothing new in that new environment it's the very same thing that's going to run in that container so you can be rest assured that they are going to run in that new environment and these docker containers are highly scalable and they are very efficient suddenly you wanted like 20 more docker containers to run the application think of adding 20 more hosts 20 more vms right how much time would it take and compared to that time the amount of time that docker containers would require to kind of scale to that amount like 20 more containers it's very less or it's minimal or negligible so it's an highly scalable and it's a very efficient uh service you can suddenly scale number of docker containers to meet any additional demand very short boot up time because it takes a it's not going to load the whole operating system and these docker containers you know they use the linux kernel and features of the kernel like c groups and name spaces to kind of segregate the processors so they can run independently any environment and it takes very less time to boot up and the data's that are stored in the containers are kind of reusable so you can have an external data volume and i can map it to the container and whatever the space that's occupied by the container and the data that the container puts in that volume they are kind of reusable you can simply remap it to another application you can kind of remap it to the next successive container you can kind of remap it to the next version of the container next version of the application you'll be launching and you don't have to go through building the data again from the scratch whatever data the container was using previously or the previous container was using that data is available for the next container as well so the volumes that the containers users are very reusable volumes and like i said it's isolated application so it kind of isolates by its nature it kind of by the way it's designed by the way it is created it isolates one container from another container meaning anytime you run applications on different containers you can be rest assured that they are very much isolated though they are running on the same host though they are running on the same laptop let's say though they are running on the same physical machine let's say running 10 containers 10 different applications you can be sure that they are well disconnected or well isolated applications now let's talk about the advantages of ecs the advantage of ecs is improved security its security is inbuilt in ecs with ecs we have something called as a container registry you know that's where all your images are stored and those images are accessed only through https not only that those images are actually encrypted and access to those images are allowed and denied through identity and access management policies iam and in other words let's say two container running on the same instance now one container can have access to s3 and the others or the rest of the others are denied access to s3 so that kind of granular security can be achieved through containers when we mix and match the other security products available in amazon like iam encryption accessing it using https these containers are very cost efficient like i've already said these are lightweight uh processes right we can schedule multiple containers on the same node and this actually allows us to achieve high density on an ec2 instance imagine an ec2 instance that that's very less utilized that's not possible with a container because you can actually dense or crowd an ec2 instance with more container in it so to best use those resources in ec2 straightforward you can just launch one application but with when we use containers you can launch like 10 different applications on the same ec2 server that means 10 different applications can actually feed on those resources available and can benefit the application and ecs not only deploys the container it also maintains the state of the containers and it makes sure that the minimum a set of containers are always running based on the requirement that's another cost efficient way of using it right and anytime an application fails and that has a direct impact on the revenue of the company and is just make sure that you're not losing any revenue because your application has failed and easies and pretty extensible services it's like this in many organizations there are majority of unplanned work because of environment variation a lot of firefighting happens when we kind of deploy the code from one or kind of move the core or redeploy the code uh in a new environment a lot of firefighting happens there right this docker containers are pretty extensible like we discussed already environment is not a concern for containers because it's going to kind of shut itself inside a docker container and anywhere the docker container can run the application will run exactly the way it performed in the past so environment is not a concern for the docker containers in addition to that ecs is easily scalable we have discussed this already and it improves it has improved compatibility we have discussed this already let's talk about the architecture of ecs like you know now the architecture of ecs is the ecs cluster itself that's group of servers running the ecs service and it integrates with docker right so we have a docker registry docker registry is a repository where we store all the docker images or the container images so it's like three components ecs is of three components one is the ecs cluster itself right when i say easy as itself i'm referring to easiest cluster cluster of servers that will run the containers and then the repository where the images will be stored right the repository where the images will be stored and the image itself so container image is the template of instructions which is used to create a container right so it's like what's the os what is the version of node that should be running and any additional software do we need so those question gets answered here so it's the template template of instructions which is used to create the containers and then the registry is the service where the docker images are stored and shared so many people can store there and many people can access or if there's another group that wants to access they can access the image from there or one person can store the image and rest of the team can access and the rest of the team can store image and this one person can pick the image from there and kind of ship it to the customer or ship it to the production environment all that's possible in this container registry and amazon's version of the container registry is ecr and there's a third party docker itself has a container registry that's docker hub ecs itself which is the the group of servers that runs those containers so these two the container image and the container registry they kind of handle docker in an image format just an image format and in ecs is where the container gets live and then it becomes and compute resource and starts to handle requests you know starts to serve the page and starts to do the batch job you know whatever your plan is with that container so the cluster of servers ecs integrates well with the familiar services like vpc vpc is known for securing vpc is known for isolating the whole environment from rest of the customers or isolating the whole environment or the whole infrastructure from the rest of the clients in your account or from the rest of the applications in your account on and on so vpc is a service that provides or gives you the network isolation ecs integrates well with vpc and this vpc enables us to launch aws resources such as amazon ec2 instance in a virtual private network that we specified this is basically what we just discussed now let's take a closer look at the ecs how does ecs work let's find answer for this question how does ecs work ecs has got a couple of components within itself so these ecs servers can run across availability zone as you can see there are two availability zones here they can actually run across availability zones and ecs has got two modes far gate mode and the ec2 mode right here we're seeing forget mode and then here we're seeing nothing that means it's an ec2 mode and then it has got different network interfaces attached to it because they need to be running in an isolated fashion right so anytime you want network isolation you need separate ip and if you want separate ip you need separate network interface card and that's what you have elastic network interface card separate elastic network interface card for all those tasks and services and this runs within an vpc let's talk about the far gate service tasks are launched using the far gate service so we will discuss about uh task what is forget now fargate is a compute engine in ecs that allows users to launch containers without having to monitor the cluster ecs is a service that manages the containers for you right otherwise managing containers will be and full-time job so easy manages it for you and if you and you get to manage ecs that's the basic service but if you want amazon to manage ecs and the containers for you we can go for fargate so farget is a compute engine in ecs that allows users to launch containers without having to monitor the ecs cluster and the tasks the tasks that we discussed the tasks has two components you see task right here so they have two components we have ecs container instance and then the container agent so like you might have guessed right now ecs container instance is actually an ec2 instance right capable of running containers not all ec2 instances can run containers so these are like specific ec2 instances that can run containers they are ecs container instances and then we have container agent which is the agent that actually binds those clusters together and it does a lot of other housekeeping work right kind of connects clusters makes sure that the version needed is present so it's all part of that agent or it's all job of that agent container instances container instances is part of amazon ec2 instance which run amazon ecs container agent pretty straightforward definition and then a container agent is responsible for communication between ecs and the instance and it also provides the status of the running containers kind of monitors the container monitors the state of the container make sure that the content is up and running and if there's anything wrong it kind of reports it to the appropriate service to fix the container on and on it's a container agent when we don't manage container agent it it runs by itself and you really don't have to do anything to make the container agent better it's already better you really won't be configuring anything in the agent and an elastic network interface car is a virtual interface network that can be connected uh to an instance in vpc so in other words elastic network interface is how the container interacts with another container and that's how the container interacts with the ec2 host and that's how the container interacts with the internet external world and a cluster a cluster is a set of ecs container instances it's not something that's very difficult to understand it's simply a group of ec2 instances that runs that ecs agent and this cluster in cluster handles the process of scheduling monitoring and scaling the request we know that ecs can scale the containers can scale how does it scale that's all monitored and managed by this ecs cluster let's talk about the companies that are using amazon ecs there are variety of companies that use ecs clusters to name a few octa users easiest cluster and octa is a product that use identity information to grant people access to applications on multiple devices at any given point of time they make sure that they have a very strong security protection so octa uses amazon ecs to run their octa application and serve their customers and abhima abhima is an a tv channel and they chose to use microservices and docker containers they already had microservices and docker containers and when they thought about a service that they can use in aws ecs was the only service that they can immediately adapt to and because in abima tv the engineers have already been using docker and docker containers it was kind of easy for them to adapt themselves to ecs and start using it along with the benefits that ecs provides previously they had to do a lot of work but now ecs does it for them all right similarly remind and ubisoft gopro are some of the famous companies that use amazon ecs and get benefited from its scalability get benefited from its cost gets benefited from its amazon managed services get benefited from the portability that ecs and the migration option that ecs provides let's talk about how to deploy a docker container on amazon acs the way to deploy docker container on ecs is first we need to have an aws account and then set up and run our first ecs cluster so in our lab we're going to use the launch wizard to run an ecs cluster and run containers in them and then task definition task definition tells the size of the container the number of the container and when we talk about size it tells how much your cpu do you need how much of memory do you need and talking about numbers you know it requires how many numbers of container you're going to launch you know is it 5 is it 10 or is it just one running all the time now those kind of information goes in the task definition file and then we can do some advanced configuration on ecs like load balancers and you know what port number you want to allow and you don't want to allow you know who gets access who shouldn't get access and what's the ip that you want to allow and deny requests from on and on and this is where we would also mention the name of the container so to differentiate one container from the other and the name of the servers you know is it an a backup job is it a web application is it an uh data container is it going to take care of your data data back end and the desired number of tasks that you want to be running all the time those details go in when we try to configure the ecs service right and then you configure cluster you put in all the security in the configure your cluster step or configure cluster stage and finally we will have an instance and bunch of containers running in that instance all right let's do a demo so here i have logged in to my amazon portal and let me switch to the appropriate region i'm going to pick not virginia north virginia look for ecs and it tells ecs as a service that helps to run and manage docker containers well and good click on it i'm in north virginia just want to make sure that i'm in the right region and go to clusters and here we can create cluster this is our far gate and this is our ec2 type launching for linux and windows environment but i'm going to launch through this walkthrough portal right this gives a lot of information here so the different steps involved here is creating a container definition which is what we're going to do right now and then a task definition and then service and finally the cluster it's a four step process so in container definition we define the image the base image we are going to use now here i'm going to launch an uh httpd or a simple http web page right so a simple httpd 2.4 image is fair enough for me and it's not a heavy application so 0.5 gigabit of a memory is enough and again it's not a heavy application so 0.25 virtual cpu is enough in our case right you can edit it based on the requirement you can always edit it and because i'm using hdpd the port mapping is already port 80 that's how the container is going to receive the request and there's no health check as if now when we want to design critical and complicated environments uh we can include health check right and this is the cpu that we have chose we can edit it and i'm going to use some bash commands to create an html page right this page says that you know amazon ecs sample app right and then it says amazon ecs sample app your application is running on a container in amazon ecs so that's the page the html page that i'm going to create or index.html so i'm going to create and put it in an appropriate location so those pages can be served from the container right if you replace this with any of your own content then it's going to be your own content ecs comes with some basic logs and these are the places where they get stored that's not the focus as of now all right so i was just saying that you can edit it and customize it to your needs we're not going to do any customization now we're just getting familiar with ecs now and the task definition name of the task definition is uh first one task definition and then we are running it in a vpc and then this is an far gate mode meaning the servers are completely handled by amazon and the task memory is 0.5 gigabit and the tar cpu is 0.25 virtual cpu name of the service is it a batch job is it an you know front end is it a back end or is it a simple copy job what's the service name of the service goes here again this you can edit it and here's a security group as of now i'm allowing 480 to the whole world if i want to restrict to a certain ip i can do that the default option for load balancing is a no load balancer but i can also choose to have a load balancer and use port 80 to map that 480 to the container put 80 right i can do that the default is no load balancer all right let's do one thing let's use load balancer let's use load balancer and port 80 that receives information on port 80 http uh what's going to be the cluster name we're in the last step what is the cluster name cluster name can be simply learn ecs demo next we're done and we can create so it's launching a cluster as you can see and it's picking the task definition file that we've created and it's using that to launch and service and then these are the log groups that we discussed and it's creating a vpc remember ecs clubs well with the vpc it's creating a vpc and it's creating two subnets here for high availability it's creating that security group port 80 allowed to the whole world and then it's putting it behind and load balancer it generally would take like five to 10 minutes so we just need to be patient and let it complete its creation and once this is complete we can simply access the servers using the load balancer url and when this is running let me actually take you to the other products or the other services that are integrated with the ecs it's getting created our service is getting created as of now ecr repository this is where all our images are stored now as of now i'm not pulling my image from ecr i'm pulling it directly from the internet docker docker hub but all custom images all custom images they are stored in this repository so you can create a repository call it app one create a repository so here's my repository so any image that i create locally or any docker image that i create locally i can actually push them push those images using these commands right here and they get stored here and i can make my ecs connect with ecr and pull images from here so they would be my custom images and as of now because i'm using a default image it's directly pulling it from the internet let's go to ec2 and look for a load balancer because we wanted to access the application from behind a load balancer right so here is a load balancer created for us and anytime i put the url so cluster is now created you see there's one service running all right let's click on that cluster here is the name of our application and here is the tasks the different containers that we are running and if you click on it we have an ip right ip of that container and it says it's running it was created at such and such time and started at such and such time and this is the task definition file that it this container uses meaning the template the details to all the version details they all come from here and it belongs to the cluster called simply learn ecs demo right and you can also get some logs container logs from here so let's go back and there are no ecs instances here because remember this is forget you're not managing any ecs instance all right so that's why you're not seeing any ecs instance here so let's go back to task and go back to the same page where we found the ip pick that ip put it in the browser and you have this sample html page running from an container so let me go back to load balancer ec2 and then under ec2 i'll be able to find a load balancer find that load balancer pick that dns name put it in the browser and now it's accessible to the load balancer url right now this url can be mapped to other services like dns this url can be embedded in any of your application if you want to make that application connect with this container now using ip is not all that advisable because these containers can die and then a new container gets created and when a new container gets created it gets a new ip right so a hard coding ip is not hardcoding dynamic ips are not advisable so you would be using load balancer and putting that url in that application that you want to make it interact with this container instance it was a wonderful experience in walking you through this ecs topic and in here we learned about what aws is and why we're using ecs and what is easiest in general what is docker in specific and we also learn about the advantages of ecs the architecture the different components of ecs and how ecs works when they're all connected together and we also looked at the companies that use ecs and their use cases and finally a lab how we can launch ecs fargate through the portal i'm very glad to walk you through this lesson about route 53. so in this section we are going to talk about basics of aws and then we're going to immediately dive into why why we need amazon route 53 and then we're going to expand and talk about the details of amazon route 53 the benefits it provides over its competitors and the different types of routing policy it has and some of amazon ralph 53's key features and we're going to talk about how to access route 53 i mean the different ways the different methods you can access route 53 and finally we're going to end with an a wonderful demo in drop 53 so let's talk about what is aws amazon web services or aws in short is a cloud provider that offers a variety of services such as a variety of id services or infrastructure services such as a compute power database content delivery and other resources that helps us to scale and grow our business and aws is hot aws is picking up aws is being adapted by a lot of customers that's because aws is easy to use even for a beginner and talking about safety the the aws infrastructure is designed to keep the data safe irrespective of the size of the data with small data be it very minimal data be it all the data that you have in terabytes and in petabytes amazon can keep it safe in their environment and the wonderful thing and the most important reason why a lot of customers move into the cloud is that the pay as you go pricing there is no long-term commitment and it's very cost effective what this means is that you're not paying for a resource that you're not using in on premises you do pay for resources you're not using a lot meaning you go and buy a server you do the estimate for the next five years and only after like three or four years you'll be hitting the peak capacity but still you would be buying that capacity before four years right and then you will gradually be you know utilizing it from you know 40 days 60 days 70 80 and then 100. so what you have done is that even though you're not using the full capacity you still have bought it and are and are paying for it from day one but in the cloud it's not like that you only pay for the resources that you use anytime you want more you scale up the resource and you you pay for the scaled up resource and anytime you want less you scale down the resource and you pay less for that scaled down resource let's talk about why amazon route 53 let's take this scenario where rachel is trying to open her web browser and the url that she hit isn't working a lot of reasons behind why the url isn't working it could be the server utilization that went high it could be it could be the memory usage that went high a lot of reasons and she starts to think is there an efficient way to scale resources according to the user requirements or is there an efficient way to kind of mask all those failures and kind of divert the traffic to the appropriate active you know active resource or active service that's running our application you always want to hide the failures right in it kind of mask the failure and direct the customer to another healthy service that's running right none of your customers would want to see a server not available or you know none of the customers your customers would want to see your service not working not impressive to them and this is tom tom is an i.t guy and he comes up with an idea and he's answering rachel yes we can scale resources efficiently using amazon rod 53 in a sense he's saying that yes we can mask the failure and we can keep the services up and running meaning we can provide more high availability to our customers with the use of route 53 and then he goes on and explains amazon drop 53 is a dns service that gives developers an efficient way to connect users to internet applications without any downtime now downtime is the key amazon route 53 helps us to avoid any downtime that customers will experience you still will have downtime in your server and your application but your customers will not be made aware of it and then rachel is kind of interested and she's like yeah that sounds interesting i want to learn more about it and tom goes on and explains the important concepts of amazon rock 53 that's everything that i'm going to explain it to you as well all right so what is amazon rat53 amazon raw 53 is a highly scalable dns or domain name system web service this service this amazon raw 53 it functions three main things or it has three main functions so the first thing is if a website needs a name route 53 registers the name for the website domain let's say you want to buy google.com you want to buy the domain name let's say you want to buy that domain name you buy that through rot53 secondly a route53 is the service that actually connects your server which is running your application or which is holding which is serving your web page so that's the service that actually route 53 is the service that connects the user to your server when they hit google.com in the browser or whatever domain name that you have purchased so you bought a domain name and the user types in yourdomainname.com and then roth53 is a service that helps the user to connect their browser to the application that's running in an ec2 instance or any other server that you are using to serve that content and not only that route 53 checks health off the resource by sending automated requests over the internet to a resource so that's how it identifies if there is any resource that has failed when i say resource i'm referring to any infrastructure failure any application level failure so it kind of keeps checking so it understands it first before the customer notices it and then it does the magic kind of shifts the connection from a one server to the other server we call it routing we will talk about that as we progress so the benefits of using route53 it's highly scalable meaning suddenly let's say the number of requests the number of people trying to access your website through that domain name that you have bought let's say it has increased ralph 53 is highly scalable right it can handle even millions and millions of requests because it's highly scalable and it's managed by amazon the same thing it's reliable it's a highly scalable it can handle large queries without the users without you interacting without the user who bought it interact with it you don't have to scale up you know when you're expecting more requests it automatically scales and it is very reliable in a sense that it's very consistent it has the ability to route the users to the appropriate application through the logic that it has it's very easy to use when we do the lab you're going to see that it's very easy easy to use you buy the domain name and then you simply map it to the application you simply map it to the server by putting in the ip or if you you can simply map it to another load balancer by putting in the load balancer url you can simply map it to another s3 bucket by simply putting the s3 bucket name or the sd bucket url it's pretty straightforward easy to set up and it's very cost effective in a way that we only pay for the service that we have used so no wastage of money here so the billing is set up in such a way that you are paying only for the amount of requests that you have received right the amount of traffic the amount of requests that you have received and couple of other things the the number of uh hosted zones that you have created right and a couple of other things it's very cost effective in such a way that you only pay for the service that you are using and it's secure in a way that access to raw 53 is integrated with identity and access management iam so you only have authorized users gain access to route 53 the trainee who just joined yesterday won't get access and the contractor or the consultant the third-party consultant you have given access or who is using your environment you can block access to that particular person because he's not the admin or he's not a privileged user in your account so only privileged users and admin gain access to route 53 through iam now let's talk about the routing policies so when you create a record in in route 53 recorder is nothing but an entry so when you do that you choose a routing policy right routing policy is nothing but it determines how route 53 responds to your queries how the dns queries are being responded right that's that's a record or that's a routing policy so the first one is a simple routing policy so we use simple routing policy for a single resource in other words simple routing allows to configure dns with no special route 53 routing it's kind of one-to-one you use an single resource that performs a given function to your domain for example if you want to simply map an url to a web server that's pretty straightforward simple routing so it routes traffic to a single resource example web server to a website and with simple routing multiple records with the same name cannot be created but multiple values can be created in the same record the second type of routing policy is failover routing so we would be using failover routing when we want to configure active passive failover if something failed right you want to fail over to the next resource which was previously the backup resource now the active resource or which was previously the backup server now it's an active server so you would be failing over to that particular resource or that particular ip if you want to do that we use failover routing so failover routing routes traffic to a resource when the resource is healthy or to a different resource when the previous resource is unhealthy in other words anytime a resource goes unhealthy i mean it does all that's needed to shift the traffic from the primary resource to the secondary resource in other words from the unhealthy resource to the healthy resource and this records can route traffic to anything from an amazon s3 bucket or you can also configure a complex tree of records now when we configure the records it will be more clear to you so as of now just understand that route 53 can route or this routing policy the failover routing policy can route traffic to amazon s3 bucket or to a website that has complex tree of records geolocation routing policy now geolocation routing just like the name says it takes that routing decision based on the geographic location of the user in other words you know when you want to roll traffic based on the location of the user if that's your primary criteria for you know sending that request to the appropriate server we will be using geo location routing so it localizes the content and presents a part or the entire website in the language of the user for example a user from us you would want to direct them to an english website and a user from german if you want to send them to the german website and a user from france you know you want to send those requests or you want to show content specific to a customer who lives in france a french website so this is if that's your condition this is the routing policy we would be using and the geographic locations are specified by either continent or by country or by state in the united states so only in the united states you can actually split it to state level and for the rest of the countries you can do it on a country level on a high level you can also do it on a continent level the next type of routing policy would be your proximity routing geoproximity routing policy when we want to route traffic based on the location of our resource and optimally shift traffic from resources in one location to resource in another location we would be using geoproximity routing so geoproximity routing routes traffic to the resources based on the geographic location of the user and the resources they want to access and it also has an option to route more traffic or less to a given resource by specifying a value known as a bias kind of weight but we also have weighted routing that's different so we've chosen different name buyers so you can send more traffic to a particular resource by having a bias on that particular routing condition and a bias expands or shrinks the size of the geographic region from which traffic is routed to a resource and then we have latency based routing just like the name says we use latency based routing if we have resources in multiple aws regions and if you want to route traffic to the region that provides the best latency at any given point of time so let's say if one single website needs to be installed and hosted on multiple aws regions then latency routing policy is what is being used it improves the performance of the users by serving the request from the aws region that provides the lowest latency so at any given point if performance is your criteria and at any given point of time irrespective of what happens in amazon infrastructure irrespective of what happens in the internet if you want to route your users to the best performing website best performing region then we would be using latency based dropping and for using latency based routing we should create latency records for the resources in multiple aws regions and then the other type of a routing policy is a multi-value routing policy where we can make route 53 to respond to dns queries with up to eight healthy records selected at random so you're not kind of loading one particular server we can define eight records and on a random basis route 53 will respond to queries from these eight records so it's not one server that gets all the requests but eight servers gets the request in a random fashion so it's multi-value routing policy and what we get by this is that we are distributing the traffic to many servers instead of just one server so multi-value routing configures rod 53 to return multiple values in response to a single or multiple dns queries it also checks the health of order sources and returns the multiple values only for the healthy resources let's say out of the eight servers we have define one server is not doing healthy it will not respond to the query with the details of the unhealthy server right so now it's gonna treat it as only seven servers in the list because one server is unhealthy and it has the ability to return multiple health checkable ip addresses to improve availability and load balancing the other type of routing policy is weighted routing policy and in here we use to route traffic or this is used to route traffic to multiple resources in a proportion that we specify so this is an weighted routing and weighted routing routes multiple resources to a single domain name or a sub domain and control the traffic that's routed to each resources so this is very useful when you're doing load balancing and testing new versions of the software so when you have a new version of the software you really don't want to send 100 of the traffic to it so you want to get customers feedback about the new software that you've launched new version or new application that you've launched so you would kind of send only 20 of the traffic to that application get customer feedback and if all is good then we would move the rest of the traffic to that new application so any software launches application launches will be using weighted routing now let's talk about the key benefits or key features of route 53 some of the key features of route 53 are a traffic flow it routes end users to the endpoint that should provide the best user experience that's what we discussed in the routing policies right it uses a routing policy a latency based routing policy and jio based routing policy and then failover routing policy so it kind of improves the user experience and the key feature the other key feature of route53 is we can buy domain names using roth53 using route53 console we can buy it from here and use it in rough 53. previously it was not the case but now we can buy it directly from amazon through route 53 and we can assign it to any resources that we want so anybody browsing that url the connection will be directed to the server in aws that runs our website a health checks it monitors health and performance of the application so it comes with an health check attached to it health check are useful to make sure that the unhealthy resources are retired right the unhealthy resources are taken away or your customers are not kind of hitting the unhealthy resources and they see an service down page or something like that uh we can have weighted round robin load balancing that's helpful in spreading traffic between several services or servers we are round robin algorithm so no one server is fully hit or no one server kind of fully absorbs all the traffic you know you can shift you can split and shift the traffic to different servers based on the weight that you would be configuring and also weighted routing also helps with a soft launch soft launch of your new application or the new version of your website there are different ways we can access amazon route 53 so you can access amazon route 53 through aws console you can also access amazon route 53 using aws sdks and we can access it using we can configure it using the apis and we can also do it through the command line interface that's linux type linux flavor aws command line interface we can also do that using windows command line windows powershell flavored command line interface as well now let's look at some of the companies that are using ros 53 so some of the famous companies that use route 53 are medium medium is an online publishing platform and it's more like a social journalism it's kind of having hybrid collection of professionals people and publications or exclusive blogs or publishers on medium it's kind of an blog website and that uses rot53 for the dns service a reddit is an social news aggregation or web content rating and discussion website that uses route 53 so these are some websites that that are accessed throughout the world and they are using roth 53 and it's highly scalable suddenly there is a new news right their website will be accessed a lot and they need to keep their service up and running all the time more availability otherwise customers will end up in a broken page and the number of customers who will be using the website will come down so it's very critical these sites these companies are very critical you know they're being highly available their page their site being highly available and the internet is very critical and crucial for them and they rely and use route 53 to meet that particular demand and airbnb is another company uh instacart cozara is another company stripe is another company that uses route53 to as their dns uh provider for their dns service they use route53 so their customers get best performance they use raw 53 so their website is highly available they use route 53 to kind of shift to the traffic between the resources so their resources are properly used with all the weighted routing the resources are properly used now let's quickly look at a demo i'm in my aws console and i'm in route 53 so let me click on rot53 so in this lab we're actually going to simulate buying a domain name and then we're going to create an s3 static website and we're going to map that website to this dns name right so the procedure is the same for mapping load balancer the procedure is the same for mapping cloudfront the procedure is the same for mapping ec2 instances as well we're picking s3 for simplicity right but our focus is actually on route53 so let's go in here and i will see if we can we'll buy a domain name here so let's first check the availability of a domain name called simply learn hyphen demo hyphen route 53 let's check its availability it is available for 12 so let me add it to cart and then come back here and then once you continue it'll ask for personal information once you give the personal information you finally check out and then it gets added to your shopping list once you pay for it amazon takes like 24 to 48 hours to make that a dns name available so the next stage would be contact details and then the third stage would be verify and purchase so once we have bought the domain name it will become available in our dns portal and i do have a domain name which i bought some time back and it's now available for me to use so i can go to hosted zone and simply start creating i can go to hosted zone and then here it's going to list all the domain names for me right click on the domain name and then click on the record set and here i can actually map elastic load balancer s3 website vpc endpoint api gateway and cloud front elastic bean stock domain names right all that gets mapped through this portal quite simple like four or five step button clicks and then it'll be done so i have an domain name bot and then i'm gonna go to s3 and i'll show you what i've done in s3 so i've created a bucket name called as a dns name let me clear the content in them so i've created a bucket and then permissions i've turned off public access blocking and then i've created and a bucket policy so this bucket is now publicly accessible and then i went on the properties and created the static website hosting right and i've pointed that this is the file that's my index file that i'm going to put or name of the file that's going to be my index file that i'm going to put in this s3 bucket so put the index file.html saved it and we're going to create a file now we're going to create an index file so this is a sample code it says amazon.53 getting started routing internet traffic to s3 bucket for your website and then a couple of other information so save it as an index.html file in my desktop so let me upload that from my desktop into this bucket so that's index.html and it's in capital i so let me go to properties and go to static website hosting and make sure that i spell it properly right it's case sensitive and then save it so now this means that my website should be running through this url and it does it's running to the static website url we're halfway through so now let me go back to route53 go back to route53 go back to hosted zones go into the domain name and then create a record set and it's going to be an alias record and i i see my s3 static website endpoint there right so click on it and create it has now created an record that's pointing my domain name to the s3 endpoint that i have created and my static website is running from it so let me test it right so let me go to the browser put the domain name in there and sure enough the domain name when my browser queried for the domain name a ros 53 returned a response saying this domain name is actually mapped to the s3 bucket static website hosting enable s3 bucket and this is the url for that static website hosting and then my browser was able to connect to that s3 bucket and download the details and show it in my browser right so it's that simple and pretty straight forward today's session is on aws elastic beanstalk so what's in it for you today we'll be discussing about what is aws why we require aws elastic bean stock what is aws elastic bean stock the advantages disadvantages the components of mean stock along with that the architecture and the companies that are primarily using the aws be installed so let's get started and first understand what is aws aws stands for amazon web services it's a cloud provider and that offers a variety of services such as compute power database storage content delivery and many other resources so we know that aws is the largest cloud provider in the market and so many services are available in the aws where you can apply the business logics and create the solutions using the cloud platforms now why aws elastic bean stock now what happened earlier and uh that whenever the developer used to create the software or the modules related to the softwares it has to be joined together to create a big application now one developer creates a module that has to be shared with another developer and if the developers are geographically separated then it has to be shared over a medium probably an internet so that is going to take some time uh it would be a difficult process and in return it uh makes the application or a software development a lender process the building of the software development lend their process so there were challenges which the developers were facing earlier and uh to overcome that uh we have the beanstalk as a service available in the aws so why aws elastic beanstalk is required uh aws elastic being stock has made the life of the developers quite easy in terms of that they can share the applications across different devices at a shorter time duration now let's understand what is aws elastic bean stock aws elastic bean stock is a service which is used to deploy and scale web applications by developers not only web application any application that is being developed by the developers this is a simple representation of the aws elastic bean stock now along with that the aws elastic bean supports the programming language the runtime environments that are java.net php node.js python rubygo and docker and in case if you're looking for any other programming language or a runtime environment then you can make a request with aws to arrange that for you now what are the advantages associated with the elastic bean stock first advantage is that it's a highly scalable service now when we talk about the scalability it means that whenever we require the resources in demand we can scale up the resources or we can scale down the resources so that is kind of a flexibility we get in terms of changing the type of resources whenever we need it and in that case the elastic bean stock is a highly scalable service now that is something which is very difficult to achieve in case of an on-prem environments because you have to plan for the infrastructure and in case if you're short of the resources within that infrastructure then you have to procure it again the second advantage associated with the beanstalk is that it's a fast and simple to begin now when we say it's fast and simple that means that you just have to focus on the development of an application building an application and then you can just deploy the application directly using the beanstalk what the beanstalk is going to do that every networking aspect is being taken care by the bean stock it deploys your application in the back end on the servers and then you can directly access your application using the url or through the ip address the third advantage is that it offers the quick deployment that is what we discussed in the fast and simple to begin as well so why it offers a quick deployment you don't have to bother about the networking concepts you just have to focus on the application development and then you can just upload your application deploy that and then you're good to go the other advantage is that it supports multi-tenant architecture when we talk about tenants or multi-tenants that means we can have a virtual environments for separate organizations or the divisions within the organizations that will be virtually isolated so likewise you can have a virtually isolated environments created on the bean stock and they can be separated used as a separate entities or a separate divisions within the organization and we know that it's a flexible service since it's a scalable then it is a flexible also now coming to the simplifies operations as an advantage now once the application is deployed using the bean stock then it becomes very easy to maintain and support that application using the beanstalk services itself and the last advantage that we can have from the beanstalk is that it's a cost efficient service the cost efficient as we know that many of the aw services are cost effective the cost optimization can be better managed using the aws mean stock as compared to if you are developing or if you are deploying any kind of an application or a solution on the on-prem servers now there are some components that are associated with the aws bean stock and it has to be created in the form of a sequence manner so aws elastic bean stock consists of few important components which are required while developing an application now what are these components these are four components one is application the second is application version the third is environment and the fourth one is the environment tier and we have to progress while deploying our applications or the softwares using the same sequence now let's understand what are the different components of the mean stock r the application it refers to a unique label which is used as a deployable code for a web application so generally you deploy your web application or you create your application and that is something which is basically uh used as a unique label then the second component is application versions so it resembles a folder which stores a collection of components such as environments versions and environment configurations so all these components are being stored using the application version the third most important component is the environment in the environment only the current versions of the applications runs now remember that elastic means stock supports multiple versions as well and using the environment you can only run the current version of the application for if you wanted to have another version of an application to be running then you have to create an another environment for that then comes the environment here and in the environment here it is basically it designates the type of application that the environment runs on now generally there are two types of environment here one is the web and the other one is the worker node and that's something which we'll be discussing later as well now let's understand how does elastic beam stock in aws works so first we have to create an application and this is a task that would be done by the developers and for that you can actually select any runtime environment or a programming language like java docker ruby gopal or python as well and once you select that environment you can develop your application using that runtime environments now after that once the application is created then you have to upload the version of an application on the aws and after that once the version is uploaded and then you have to launch your environment so just have to click on the buttons that's it nothing more you have to do once the environment is launched then you can actually view that environment using a web url or using the ip address now what happens in that case is when you launch an environment in the back end the elastic bean stock runs automatically runs an ec2 instance and using a metadata the mean stock deploys our application within that ec2 instance that is something which you can look into the ec2 dashboard as well so you don't have to take care of the security groups you don't have to take care of the ip addressing and even you don't have to login into the instance and deploy your application it would be done automatically by the bean stock it's just that you just have to monitor the environment and the statistics will be available there itself in the beanstalk dashboard otherwise you can view those statistics in the cloudwatch logs as well now in case if you wanted to update any kind of a version then you just upload a new version and then just deploy that and then monitor your environment so these are the essentials to create a local applications for any platform whether it's a node.js python etc these are the things that you have to actually take care and this is the sequence you have to follow while creating an environment so you can say that it's a four steps creation of a deployment of your application that's it now after users upload their versions the configuration is automatically deployed with a load balancer yes and with the load balancer that means you can access the applications using the load balancer dns also and apart from load balancer if you wanted to put any other feature that includes the auto scaling for example if you wanted to create your ec2 instances where the application will be deployed within the virtual private cloud or in a particular subnet within the vpc all those features that are available and you can select them using the mean stock itself you don't have to move out to the vpc you don't have to actually go to the ec2 dashboard and select all those separately everything would be available within the beanstalk dashboard so that's what it says in the presentation that after creating an application the deploy service can be specifically accessed using the url so once the environment is created there will be a url defined now you can put a url name also that is something which you wanted to put for your application you can define that you can check for the availability of that url and then you have to use that url to access your application or the browser now once it is done then in the monitor environment it says the environment is monitored provided capacity provisioning load balancing auto scaling and multi features all those features are available there itself in the mean stock now let's understand the architecture of aws elastic green stock now there are two types of environments that you have to select you can select one is the web server environment and the other one is the worker environment so based on the client requirement bean stock gives you two different types of environment that you have to select generally the web server environment is the front end facing that means the client should be accessing this environment directly using a url so mostly web applications are deployed using that environment the worker environment is the back-end applications or on the micro apps which are basically required to support the running of the web applications now it depends on the client requirement what kind of an environment you wanted to select now in the web server environment it only handles the http request from the clients so that's why we use the web server environment mostly for the web applications or any application which works on the http https requests so it's not only the http you can use the https as well the worker environment it process background task and minimizes the consumption of resources so again it is just like a kind of a micro service or an application services that are running in the back end to support the web server environment now coming to the understanding of the aws main stock so this is how the architecture of the aws bin stock is designed and you can refer to that image also now in the web server environment let's say if we select a web server environment and it says that if the application receives client request the amazon route 53 sends this request to the elastic load balancer now obviously we discussed here that the web server environment is primarily an environment which receives the http request it's a kind of a client-facing environment now if the application receives a client request amazon from the amazon note 53 this route 53 is a service which is primarily used for dns mapping it's a global service and it may route you can route the traffic from the route 53 matching your domains towards the load balancer and from the load balancer you can point that traffic to the web server environment obviously the web server environment is nothing it's just the ec2 instances that would be running in the back end now here in the diagram you can see that there are two web server environments and they are created in the auto scaling group that means there is some kind of scaling options that are defined as well and these instances are created in an availability zone or they can be created in a different availability zone also for the redundancy as well and these web application servers are further connected to your databases which primarily will be in a different security groups probably it can be an rds database also so all these functionalities all these features are basically available on the elastic mean stock dashboard itself now what happens in that case is if the application receives client requests amazon route 53 send these requests to the load balancer later the load balancer shares those requests among the ec2 instances how does that happen it happens using a predefined algorithm the equal distribution of a load is distributed to both the ec2 instances or n number of ac2 instances running in the availability zone now in the availability zones every ec2 instance would have its own security group they can have a common security group also they can have their own security group as well now after the security group the load balancer is then connected to the amazon ec2 instance which are part of the auto scaling group so that's something which we have discussed already now this auto scaling group is would be defined from the beanstalk itself and there will be some scaling options that will be created it could be a possibility that it might be the minimum number of instances that would be running as of now and based on the threshold defined it may increase the number of ec2 instance and the load balancer will keep on distributing the load to as many instances that will be created inside the availability source obviously there will be an internal health check that the load balancer will be first doing before distributing the real time traffic to this instances created by the mean stock now what does auto scaling group does it automatically starts the additional ec2 instance to accommodate increasing load on your application that's something which we know that and also it monitors and scales instances based on the workload as well so depends on what kind of a scaling threshold you have defined in the auto scaling groups and when the load of an application decreases the ec2 instance will also be decreased so whenever we talk about the auto scaling generally it comes in our mind is that we scale up the resources that means we it increases the ec2 instances in the auto scaling you might have the scale down option also scaled down policy also created in which if the load minimizes it can terminate the additional ec2 instances as well so that is something which will be automatically managed all these features can be achievable using the elastic bean stock and with this feature accommodated it gives you the better cost optimization in terms of managing your resources now it says that elastic bean stock has a default security group and the security group acts as a firefall for the instances now here in this diagram it says about the security group auto scaling also you might create it in a default vpc also you might create it in your custom vpc also where you can have the additional level of security is also created you can have the nacl's knuckles also defined here before the security groups so that would give you the additional filtering option or the firewall option now it says that with these groups with these security groups it allows establishing security groups to the database server as well so every database would also have its own security group and the connection can be created between the web servers environment that is created by the beanstalk to the database security groups as well now let's discuss about the worker environment now understanding the worker environment what happens is that the client the web server environment is the client facing the client sends a request for an access to the web server and in this diagram the web server further sends it to the sqs which is a simple queue service and the queue service sends it to the worker environment and then whatever the worker environment is created for doing some kind of a processing or some kind of an application that is running in the backend that environment initiates and then send back the results to this sqs and vice versa so let's understand the architecture of aws elastic bean stock with the worker environment so when a worker environment here is launched aws elastic mean stock installs the server on every ec2 instance so that is in the case of a web server environment also and later the server passes the request to the simple queue service now this service is an asynchronous service instead of a simple queue service you can have other services also it is not necessary that you need to have the sqs also this is an example that we are discussing about and the sqs shares those message via a post request to the http path over the booker environment and there are many case studies also with respect to this kind of an environment that is being created that has been done on many customers and you can search for these kind of a case studies available on the internet now the worker environment executes the task given by the sqs with the http response after the operation is completed now here what happens is a quick recap the client request for an access of an application to a web server using an http request the web server passes that request to the queue service the q service shares the message with the worker probably a worker might be the manual worker and generally it's an automated worker so it would be shared via the worker environment only and the worker sends back the response with the http response back to the queue that response can be viewed directly from the queue service by the client using the web server so this is one of the example likewise as i said that there can be many other examples also where you can have the worker environments defined now what are the companies that are using the elastic bean stock these are few of the companies that are primarily using on a zillow jelly button games then you have league of women waters ebury these are some of the few listed companies and obviously you search on the aws site and you'll find many more organizations that are using the elastic bean stock primarily for deploying their applications now the next thing is to go with the practicals that how actually we use the elastic bean stock so let's look into the demo using the aws elastic bean stock now first you have to login into the aws console and i'm sure that you might be having the accounts created or you can use the im credentials as well and then you have to select the region also now i am in the north virginia region likewise you can select any of the regions that are listed here now click on the services and you have to search for the elastic bean stock you can find the elastic bean stock under the compute section so here itself you'll find the elastic bean stock as a service now open this service and there it will give you an option to create an environment you have to specifically select an environment probably a worker environment or a web service environment so let's wait for the service to open so we have the dashboard now available with us this is how the elastic bean stock looks and this is the symbol representation of a bean stock now what you have to do is we have to click on get started and that will load and you have to create a web app so instead of creating a web app what we'll do we'll create a new application so just click on create a new application put an application name let's say we put something like x y z you can put any description to your application let's say it's a demo app and click on create now it says you have to select an environment now the environment the application name xyz is created you just have to select an environment so click on create one now and it is going to ask you that what kind of an environment here you wanted to select so as we discussed that there are two types of environments one is the web server and the other one is the current one let's look into it what is defined by the aws aw says that it has two types of environment tiers to support different types of web applications web servers are standard applications that listen for and then process http request typically over port number 80. workers are specialized application that have a background processing task that listens for message on an amazon sqs queue workers application post those messages to your application by using the http response so that's what we saw in the case of the beanstalk slides also now the usability of a worker environment can be anything now we'll do a demo for creating a web server environment so just click on select and you we have the environment name created now we can define our own domain it ends with the region dot elastic beanstalk.com let's say i look for a domain which is xyz only that's the environment name now i'll check for the availability whether that domain name is available with us or not and it says we don't have that domain name so probably i'll try to make it with some other name and let's look for the availability xyz abc and it says yes it is available now once i deploy my application i would be able to access the application using this complete dns so you can put a description it's a demo app that we are creating and then you have to define a platform as well now these are the platforms that are supported by the aws let's say i wanted to run a node.js environment so i'll just click on the node.js platform the application codes is something which is basically developed by the developers and you can upload the app application right now or you can do that later as well once the environment is ready now either you can select to create an environment if you wanted to go with all the default settings otherwise if you wanted to customize it more you can click on configure more options so let's click on configure more options and here you would be able to define various different features like the type of an instance for example what kind of an ec2 instance or a server that should be running so that the bin stock can deploy our applications over it if you wanted to modify just click on a modify button and here you can modify your instances with respect to the storage as well now apart from that if you wanted to do some modification in the case of monitoring in the case of databases in the case of security or in the case of a capacity let's look into the capacity so here you can actually do the modification so in the capacity you can select the instance type also by default it is t2.micro but in case if your application requires a larger type of an instance then you can actually go for the instance type as well similarly you can define your emi ids also because obviously for the application to run you would require the operating system also so you can select that particular ami id for your operating system as well let's cancel that likewise you have many other features that you can actually define here from the dashboard and you don't have to go to the ec2 dashboard to do the modifications now let's go and create an environment let's assume that we are going with the default configuration so this is going to create our environment the environment is being created and you can get the environment and the logs defined in the dashboard itself so you'll see that the beanstalk environment is being initiated the environment is being started and in case if there would be any errors or if it is deployed correctly you'll get all the logs here itself now the environments are basically color coded so there are different color codings that are defined if you get the environment in a green color that means everything is good to go so here you can see that it has created an elastic ip it has checked the health of the environment now it has created the security groups and that would be an auto security groups created by the bean stock and the environment creation has been started you can see that elastic bean stock has amazon s3 storage bucket for your environment data as well this is the url through which you will be accessing the environment but right now we cannot do that since the environment is being created let's click on the application name and here you can see that it is in a gray color that means right now the build is being done it is being created once it will be successfully created it should change to the green color and then we will be able to access our environment using the url now if i move to the ec2 instances and see in the ec2 dashboard if i see whether the instance is being created by the bean stock or not so let's see and let's see what are the differences in terms of creating an instance manually and getting it created from the bean stock so click on the easy to let's go to the old ec2 experience that's what we are familiar with and let's see what's there in the dashboard so here you can see one running instance let's open that and the xyz environment which was created from the beanstalk is being initiated the instance is being initiated and that is something which is being done by the mean stock itself we have not gone to the dashboard and created it manually now in the security groups if you see that here the aws mean stock security groups are defined it has the elastic ips also defined so everything is being created by the beanstalk itself right now let's go back to the beanstalk and let's look into the status of our environment whether the color coding has been changed from gray to green or not and here you can see the environment is successfully created and we have that environment colored in green we'll access the environment and it says it's a web server environment its platform is node.js running on 64-bit amazon linux ami and it says samples app sample application health status is okay now the other thing is that if you do not want to use the web console the management console to access the main stock then the beanstalk offers you the elastic beanstalk cli as well so you can install the command line interface and then you have the command references cli command references that you can actually play with and get your applications deployed using the bean stock itself so this is one of the sample cli commands that you can actually look into now let's look into the environment let's click on the environment and we'll be represented with the url it says health is okay these are the logs that you have to follow in case if there are any issues the platform is nodejs that is what we selected now the next thing is you just have to upload and deploy your applications so just click on upload and deploy select the version label or the name select file and wherever your application is hosted at just select that upload it and deploy your application you'll see that the like your environment is created similarly your application will be deployed automatically on the instance and from this url you will be able to view the output it is as simple as just like you have to follow these four steps now let's see whether the node.js environment is running on our instance before deploying an application so we'll just click on this url since the beanstalk has already opened up the security groups or http port 80 for all we can actually view that output directly from the url so we have the node.js running that's visible here and after that you just have to upload and deploy your application and then from that url you can get the output now this url you can map it with the root 53 service so using the root 53 dns services the domain names can be pointed to the elastic bean stock url and from there it can be pointed to the applications that are running on the ec2 instance whether you wanted to point it to the url directly using the mean stock you can do that otherwise as we saw in the slides you can use the root 53 pointer to the load balancer and then point it to the instances directly also once it is created by the mean stock so that was the demo guys with respect to the bean stock and how we can actually run the environments apart from that the operational task like system operations you can manage all these things from the environment dashboard itself so you have the configurations you have the logs you can actually check the health status of your environment you can do the monitoring and you can actually get the alarms and the events here so let's say if i wanted to if i wanted to see the logs i can request for the logs here itself and i will be represented with the full log report and i can now download that log file and i can view the locks so it's in the so we have this bundle locks in the zip file all right so if you want to see some kind of uh logs with respect to elastic bean stock activity it's in the form of a notepad and here you can see what all configurations the beanstalk has done on your environment on your instance similarly you can go for the health monitoring alarms events and all those things if getting your learning started is half the battle what if you could do that for free visit skill up by simply learn click on the link in the description to know more hi this is the fourth lesson of the aws solutions architect course migrating to the cloud doesn't mean that resources become completely separated from the local infrastructure in fact running applications in the cloud will be completely transparent to your end users aws offers a number of services to fully and seamlessly integrate your local resources with the cloud one such service is the amazon virtual private cloud this lesson talks about creating virtual networks that closely resemble the ones that operate in your own data centers but with the added benefit of being able to take full advantage of aws so let's get started [Music] in this lesson you'll learn all about virtual private clouds and understand their concept you'll know the difference between public private and elastic ip addresses you'll learn about what a public and private sudden there is and you'll understand what an internet gateway is and how it's used you'll learn what route tables are and when they are used you'll understand what and that gateway is we'll take a look at security groups and their importance and we'll take a look at network acls and how they're used in amazon vpc we'll also review the amazon vpc best practices and also the costs associated with running a bpc in the amazon cloud welcome to the amazon virtual private cloud and subnet section in this section we're going to have an overview of what amazon vpc is and how you use it and we're also going to have a demonstration of how to create your own custom virtual private cloud we're going to look at ip addresses and the use of elastic ip addresses in aws and finally we'll take a look at subnets and there'll be a demonstration about how to create your own subnets in an amazon vpc and here are some other terms that are used in vpcs there's subnets root tables elastic ip addresses internet gateways nat gateways network acls and security groups and in the next sections we're going to take a look at each of these and build our own custom vpc that we'll use throughout this course amazon defines a vpc as a virtual private cloud that enables you to launch aws resources into a virtual network that you've defined this virtual network closely resembles a traditional network that you'd operate in your own data center but with the benefits of using the scalable infrastructure of aws a vpc is your own virtual network in the amazon cloud which is used as the network layer for your ec2 resources and this is a diagram of default vpc now there's a lot going on here so don't worry about that what we're going to do is break down each of the individual items in this default vpc over the coming lesson but what you need to know is that a vpc is a critical part of the exam and you need to know all the concepts and how it differs from your own networks throughout this lesson we're going to create our own vpc from scratch which you'll need to replicate at the end of this so you can do well in the exam each vpc that you create is logically isolated from other virtual networks in the aws cloud it's fully customizable you can select the ip address range create subnets configure route tables set up network gateways define security settings using security groups and network access control lists so each amazon account comes with a default vpc that's pre-configured for you to start using straight away so you can launch your ec2 instances without having to think about anything we mentioned in the opening section a vpc can span multiple availability zones in a region and here's a very basic diagram of a vpc it isn't this simple in reality and as we saw in the first section here's the default amazon vpc which looks kind of complicated but what we need to know at this stage is that cidr block for the default vpc is always a 16 subnet mask so in this example it's 172.31.0.0.16. what that means is this vpc will provide up to 65536 private ip addresses so in the coming sections we'll take a look at all of these different items that you can see on this default vpc but why wouldn't you just use the default vpc well the default vpc is great for launching new instances when you're testing aws but creating a custom vpc allows you to make things more secure and you can customize your virtual network as you can define your own ip address range you can create your own subnets that are both private and public and you can tighten down your security settings by default instances that you launch into a vpc can't communicate with your own network so you can connect your vpcs to your existing data center using something called hardware vpn access so that you can effectively extend your data center into the cloud and create a hybrid environment now to do this you need a virtual private gateway and this is the vpn concentrator on the amazon side of the vpn connection then on your side in your data center you need a customer gateway which is either a physical device or a software application that sits on your side of the vpn connection so when you create a vpn connection a vpn tunnel comes up when traffic is generated from your side of the connection vpc peering is an important concept to understand appearing connection can be made between your own bpcs or with a vpc in another aws account as long as it's in the same region so what that means is if you have instances in vpca they wouldn't be able to communicate with instances in vpc b or c unless you set up appearing connection peering is a one-to-one relationship a vpc can have multiple peering connections to other vpcs but and this is important transitive peering is not supported in other words vpca can connect to b and c in this diagram but c wouldn't be able to communicate with b unless they were directly paired also vpcs with overlapping cidrs cannot be paired so in this diagram you can see they all have different ip ranges which is fine but if they had the same ipo ranges they wouldn't be able to be paired and finally for this section if you delete the default vpc you have to contact aws support to get it back again so be careful with it and only delete it if you have good reason to do so and know what you're doing this is a demonstration of how to create a custom vpc so here we are back at the amazon web services management console and this time we're going to go down to the bottom left where the networking section is i'm going to click on the pc and the vpc dashboard will load up now there's a couple of ways you can create a custom vpc there's something called the vpc wizard which will build vpcs on your behalf from a selection of different configurations for example a vpc with a single public subnet or a bpc with public and private subnets now this is great because you click a button type in a few details and it does the work for you however you're not going to learn much or pass the exam if this is how you do it so we'll cancel that and we'll go to your vpcs and we'll click on create a vpc and we're presented with the create the vpc window so let's give our vpc a name i'm going to call that simply learn underscore vpc and this is the kind of naming convention i'll be using throughout this course next we need to give it the cidr block or the classless interdomain routing block so we're going to give it a very simple one 10.0.0 and then we need to give it the subnet mask so you're not allowed to go larger than 15. so if i try to put 15 in it says no not gonna happen for reference subnet mask of 15 would give you around 131 000 ip addresses and subnet 16 will give you 65 536 which is probably more than enough for what we're going to do next you get to choose the tenancy there's two options default and dedicated if you select dedicated then your ec2 instances will reside on hardware that's dedicated to you so your performance is going to be great but your cost is going to be significantly higher so i'm going to stick with default and we just click on yes create it'll take a couple of seconds and then in our vpc dashboard we can see our simply learn vpc has been created now if we go down to the bottom here to see the information about our new vpc we can see it has a root table associated with it which is our default route table so there it is and we can see that it's only allowing local traffic at the moment we go back to the vpc again we can see it's been given a default network acl and we'll click on that and have a look and you can see this is very similar to what we looked at in the lesson so it's allowing all traffic from all sources inbound and outbound now if we go to the subnet section and just widen the vpc area here you can see there's no subnets associated with the vpc we just created so that means we won't be able to launch any instances into our vpc and to prove it i'll just show you we'll go to the ec2 section so this is a glimpse into your future this is what we'll be looking at in the next lesson and we'll just quickly try and launch an instance we'll select any instance it doesn't matter any size not important so here the network section if i try and select simply learn vpc it's saying no subnets found this is not going to work so we basically need to create some subnets in our vpc and that is what we're going to look at in the next lesson now private ip addresses are ip addresses that are not reachable over the internet and they're used for communication between instances in the same network when you launch a new instance is given a private ip address and an internal dns hostname that resolves to the private ip address of the instance but if you want to connect to this from the internet it's not going to work so then you'd need a public ip address which is reachable from the internet you can use public ip addresses for communication between your instances and the internet each instance that receives a public ip address is also given an external dns hostname public ip addresses are associated with your instances from the amazon pool of public ip addresses when you stop or terminate your instance the public i p address is released and a new one is associated when the instance starts so if you want your instance to retain this public ip address you need to use something called an elastic ip address an elastic ip address is a static or persistent public ip address that's allocated to your account and can be associated to and from your instances as required an elastic ip address remains in your account until you choose to release it there is a charge associated with an elastic ip address if it's in your account but not actually allocated to an instance this is a demonstration of how to create an elastic ip address so we're back at the amazon web services management console we're going to head back down to the networking vpc section and we'll get to the vpc dashboard on the left hand side we'll click on elastic ips now you'll see a list of any elastic ips that you have associated in your account and remember any the elastic ip address that you're using that isn't allocated to something you'll be charged for so i have one available and that is allocated to an instance currently so we want to allocate a new address and it reminds you that there's a charge if you're not using it i'm saying yes allocate and it takes a couple of seconds and there's our new elastic ip address now we'll be using this ip address to associate with the nat gateway when we build that aws defines a subnet as a range of ip addresses in your bpc you can launch aws resources into a subnet that you select you can use a public subnet for resources that must be connected to the internet and a private subnet for resources that won't be connected to the internet the netmask for the default subnet in your vpc is always 20 which provides up to 4096 addresses per subnet and a few of them are reserved for aws use a vpc can span multiple availability zones but the subnet is always mapped to a single availability zone this is important to know so here's our basic diagram which we're now going to start adding to so we can see the virtual private cloud and you can see the availability zones and now inside each availability zone we've rated a subnet now you won't be able to launch any instances unless there are subnets in your vpc so it's good to spread them across availability zones for redundancy and failover purposes there's two different types of subnet public and private you use a public subnet for resources that must be connected to the internet for example web servers a public subnet is made public because the main route table sends the subnets traffic that is destined for the internet to the internet gateway and will touch on internet gateways next private subnets are for resources that don't need an internet connection or that you want to protect from the internet for example database instances so in this demonstration we're going to create some subnets a public and a private subnet and we're going to put them in our custom vpc in different availability zones so we'll head to networking and vpc wait for the vpc dashboard to load up we'll click on subnets we'll go to create subnet and i'm going to give the subnet a name so it's good to give them meaningful names so i'm going to call this first one for the public subnet 10.0.1.0 and i'm going to put this one in the us east one b availability zone and i'm going to call that simply learn public so it's quite a long name i understand but at least it makes it clear for what's going on in this example so we need to choose a vpc so we obviously want to put it in our simply learn vpc and i said i wanted to put it in us east 1b i'm using the north virginia region by the way so we click on that then we need to give it the cidr block now as i mentioned earlier when i typed in the name that's the range i want to use and then we need to give it the subnet mask and we're going to go with 24 which should give us 251 addresses in this range which obviously is going to be more than enough if i try and put a different value in that's unacceptable to amazon it's going to say it's going to give me an error and tell me not to do that let's go back to 24 and click on the cut and paste this by the way just because i need to type something very similar for the next one click create and it takes a few seconds okay so there's our new subnet and i just widen this you can see so that's the ip range that's the availability zone it's for simply learn and it's public so now we want to create the private so put the name in i'm going to give the private the ip address block of that i'm going to put this one in usd 1c and it's going to be the private subnet obviously i want it to be in the same vpc where the bit is the zone of us east 1c and we're going to give it 10.0.2.0. 24. and we'll click yes create again it takes a few seconds okay so we sort by name so there we are we can see now we've got our private subnet and our public server let me just type in simply then there we are so now you can see them both there and you can see they're both in the same vpc simply learn vpc now if we go down to the bottom you can see the root table associated with these vpcs and you can see that they can communicate with each other internally but there's no internet access so that's what we need to do next in the next lesson you're going to learn about internet gateways and how we can make these subnets have internet access welcome to the networking section in this section we're going to take a look at internet gateways root tables and nat devices and we'll have a demonstration on how to create each of these aws vpc items so to allow your vpc the ability to connect to the internet you need to attach an internet gateway and you can only attach one internet gateway per vpc so attaching an internet gateway is the first stage in permitting internet access to instances in your vpc now here's our diagram again and now we've added the internet gateway which is providing the connection to the internet to your vpc but before you can configure internet correctly there's a couple more steps for an ec2 instance to be internet connected you have to adhere to the following rules firstly you have to attach an internet gateway to your vpc which we just discussed then you need to ensure that your instances have public ip addresses or elastic ip addresses so they're able to connect to the internet then you need to ensure that your subnet's root table points to the internet gateway and you need to ensure that your network access control and security group rules allow relevant traffic to flow to and from your instance so you need to allow the rules to let in the traffic you want for example http traffic after the demonstration for this section we're going to look at how route tables access control lists and security groups are used in this demonstration we're going to create an internet gateway and attach it to our custom vpc so let's go to networking bpc bring up the vpc dashboard and on the left hand side we click on internet gateways so here's a couple of internet gateways i have already but i need to create a new one so create internet gateway i'll give it a name which is going to be simply learn internet gateway igw and i'm going to click create so this is an internet gateway which will connect a vpc to the internet because at the moment our custom vpc has no internet access so there is created simply then i gw but this state is detached because it's not attached to anything so let me try and attach it to a vpc and it gives me an option of all the vpcs that have no internet gateway attached to them currently so i only have one which is simply then bpc yes attach now you can see our vpc has internet attached and you can see that down here so let's click on that and it will take us to our vpc but before any instances in our vpc can access the internet we need to ensure that our subnet root table points to the internet gateway and we don't want to change the main route table we want to create a custom route table and that's what you're going to learn about next a route table determines where network traffic is directed it does this by defining a set of rules every subnet has to be associated with a route table and a subnet can only be associated with one route table however multiple subnets can be associated with the same route table every vpc has a default route table and it's good practice to leave this in its original state and create a new route table to customize the network traffic routes associated with your vpc so here's our example and we've added two route tables the main route table and the custom route table the new route table or the custom route table will tell the internet gateway to direct internet traffic to the public subnet but the private subnet is still associated to the default route table the main route table which does not allow internet traffic to it all traffic inside the private subnet is just remaining local in this demonstration we're going to create a custom route table associated with our internet gateway and associate our public subnet with it so let's go to networking and vpc the dashboard will load and we're going to go to route tables now our vpc only has its main route table at the moment the default one it was given at the time it was created so we want to create a new route table and we want to give it a name so we're going to call it simplylearn i'm going to call it root table rtb for sure and then we get to pick which vpc we want to put it in so obviously we want to use simplylearn vpc so we click create which will take a couple of seconds and here you are here's our new route table so what we need to do now is change its route so that it points to the internet gateway so if we go down here to roots at a minute you can see it's just like our main route table it just has local access so we want to click on edit and we want to add another route so the destination is the internet which is all the zeros and our target and we click on this it gives us the option of our internet gateway which we want to do so now we have internet access to the subnet sorry to this route table and we click on save save was successful so now we can see that as well as local access we have internet access now at the moment if we click on subnet associations you do not have any subnet associations so basically both both our subnets the public and private subnets are associated with the main route table which doesn't have internet access so we want to change this we'll click on edit and we want our public subnet to be associated with this root table let's click on save so it's just saving that so now we can see that our public subnet is associated with this route table and this route table is associated with the internet gateway so now anything we launch into the public subnet will have internet access but what if we wanted our instances in the private subnet to have internet access well there's a way of doing that with a nat device and that's what we're going to look at in the next lecture you can use a nat device to enable instances in a private subnet to connect to the internet or other aws services but prevent the internet from initiating connections with the instances in the private subnet so we talked earlier about public and private subnets to protect your assets from being directly connected to the internet for example your web server would sit in the public subnet in your database in the private subnet which has no internet connectivity however your private subnet database instance might still need internet access or the ability to connect to other aws resources if so you can use a network address translation device for a nat device to do this and that device forwards traffic from your private subnet to the internet or other aws services and then sends the response back to the instances when traffic goes to the internet the source ip address of your instance is replaced with the nat device address and when the internet traffic comes back again then that device translates the address to your instance's private ip address so here's our diagram which is getting ever more complicated and if you look in the public subnet you can see we've now added and that device and you have to put that devices in the public subnet so that they get internet connectivity aws provides two kinds of nat devices and that gateway and a nat instance aws recommends a nat gateway as it's a managed service that provides better availability and bandwidth than that instances each nat gateway is created in a specific availability zone and is implemented with redundancy in that zone and that instance is launched from a nat ami and amazon machine image and runs as an instance in your vpc so it's something else you have to look after whereas in that gateway being a fully managed service means once it's installed you can pretty much forget about it and that gateway must be launched into a public subnet because it needs internet connectivity it also needs an elastic ip address which you can select at the time of launch once created you need to update the route table associated with your private subnet to point internet bound traffic to the nat gateway this way the instances in your private subnets can communicate with the internet so if you remember back to the diagram when we had the custom route table which was pointed to the internet gateway now we're pointing our main route table to the nat gateway so the private subnet also gets internet access but in a more secure manner welcome to the create a nat gateway demonstration where we're going to create a nat gateway so that the instances in our private subnet can get internet access so we'll start by going to networking and vpc and the first thing we're going to do is take a look at our subnets and you'll see why shortly so here are our simply learned subnets so this is the private subnet that we want to give internet access but if you remember from the section that gateways need to be placed in public subnets so i'm just going to copy the name of this subnet id for the public subnet and you'll see why in a moment so then we go to nat gateways on the left hand side and we want to create a new nat gateway so we have to put a subnet in there so we want to choose our public subnet as you can see it truncates a lot of the subnet names on this option so it's a bit confusing so we know that we want to put it in our simply learn vpc in the public subnet but you can see it's truncated so it's actually this one at the bottom but what i'm going to do is just paste in the subnet id which i copied earlier so there's no confusion then we need to give it an elastic ip address now if you remember from the earlier demonstration we created one so let's select that but if you hadn't allocated one you could click on the create new eip button so we'll do that okay so it's telling me my nat gateway has been created and in order to use your netgateway ensure that you edit your route table to include a route with a target of and then on that gateway id so it's given us the option to click on our edit route tables so we'll go straight there now here's our here's our route tables now here's the custom route table that we created earlier and this is the default the main routeover which was created when we launched our when we created our vpc so we should probably give this a name so that we know what it is so let me just call this simplylearn rtb main so now we know that's our main route table so if you take a look at the main route table and the subnet associations you can see that our private subnet is associated with this table so what we need to do is put a route in here that points to the nat gateway so if we click on routes and edit and we want to add another route and we want to say that all traffic can either go to the simply then internet gateway which we don't want to do we want to point it to our nat instance which is this nat id here and we click save so now any instances launched in our private subnet will be able to get internet access via on that gateway welcome to the using security groups and network acl section in this section we're going to take a look at security groups and network acls and we're going to have a demonstration on how we create both of these items in the amazon web services console a security group acts as a virtual firewall that controls the traffic for one or more instances you add rules to each security group that about traffic to or from its associated instances basically a security group controls the inbound and outbound traffic for one or more ec2 instances security groups can be found on both the ec2 and vpc dashboards in the aws web management console we're going to cover them here in this section and you'll see them crop up again in the ec2 lesson and here is our diagram and you can see we've now added security groups to it and you can see that ec2 instances are sitting inside the security groups and the security groups will control what traffic flows in and out so let's take a look at some examples and we'll start with a security group for a web server now obviously a web server needs http and https traffic as a minimum to be able to access it so here is an example of the security group table and you can see we're allowing http and https the ports that are associated with those two and the sources and we're allowing it from the internet we're basically allowing all traffic to those ports and that means any other traffic that comes in on different ports would be unable to reach the security group and the instances inside it let's take a look at an example for a database server security group now imagine you have a sql server database then you would need to open up the sql server port so that people can access it and which is port 1433 by default so we've added that to the table and we've allowed the source to come from the internet now because it's a windows machine you might want rdp access so you can log on and do some administration so we've also added rdp access to the security group now you could leave it open to the internet but that would mean anyone could try and hack their way into your box so in this example we've added a source ip address of 10.0.0.0 so only ipa ranges from that address can rdp to the instance now there's a few rules associated with security groups by default security groups allow all outbound traffic so if you want to tighten that down you can do so in a similar way to you can define the inbound traffic security group rules are always permissive you can't create rules that deny access so you're allowing access rather than denying it security groups are stateful so if you send a request from your instance the response traffic for that request is allowed to flow in regardless of the inbound security group rules and you can modify the rules of a security group at any time and the rules are applied immediately welcome to the create security group demonstration where we're going to create two security groups one the host db servers and one the host web servers now if you remember from the best practices section it said it was always a good idea to tear your applications into security groups and that's exactly what we're going to do so we go to networking and vpc to bring up the vpc dashboard on the left hand side under security we click on security groups now you can also get to security groups from the ec2 dashboard as well so here's a list of my existing security groups but we want to create a new security group and we're going to call it simply learn web server sg security group and we'll give the group name as the same and our description is going to be simply learn web servers security groups okay and then we need to select our vpc now it defaults to the default vpc but obviously we want to put it in our simply learn vpc so we click yes create takes a couple of seconds and there it is there's our new security group now if we go down to the rules the inbound rules you can see there are none so by default a new security group has no inbound rules but what about outbound rules if you remember from the lesson a new security group by default allows all traffic to be outbound and there you are all traffic has destination of everywhere so all traffic is allowed we're going to add some rules so let's click on inbound rules click on edit now this is going to be a web server so if we click on the drop down we need to give it http so you can either choose custom tcp rule and type in your own port ranges or you can just use the ones they have for you so http this pre-populates the port range and then here you can add the source now if i click on it it's giving me the option of saying allow access from different security groups so you could create a security group and say i only accept traffic from a different security group which is a nice way of securing things down you could also put in here just your ip address so that only you could do http requests to the instance but because it's a web server we want people to be able to see our website otherwise it's not going to be much use so we're going to say all traffic so all source traffic can access our instance on port http 80. i want to add another rule because we also want to do https which is hiding from me there we are and again we want to do the same and also because this is going to be a linux instance we want to be able to connect to the linux instance to do some work and configuration so we need to give it ssh access and again it would be good practice to tie it down to your specific ip or an ip range but we're just going to do all for now and then we click on save and there we are there we have our ranges so now we want to create our security group for our db servers so let's click create security group and then we'll go through and give it a similar name simply learn db servers sg and the description is going to be simply learn db servers security group and our vpc is obviously going to be simply learn vpc so let's click yes create wait a few seconds and here's our new security group as you can see it has no inbound rules by default and outbound rules allow all traffic so this is going to be a sql server database server and so we need to allow sql server traffic into the instance so we need to give it microsoft sql port access now the default port for microsoft sql server is 1433. now in reality i'd probably change the port the sql server was running on to make it more secure but we'll go over this for now and then the source so we could choose the ipa ranges again but what we want to do is place the db server in the private subnet and allow the traffic to come from the web server so the web server will accept traffic and the web server will then go to the database to get the information it needs to display on its web on the website or if people are entering information into the website we want the information to be stored in our db server so basically we want to say that this the db servers can only accept sql server traffic from the web server security group so we can select the simply then web server security group as the source traffic for microsoft sql server data so we'll select that now our sql server is obviously going to be a windows instance so from time to time we might not we might need to log in and configure it so we want to give rdp access now again you would probably put a specific ip range in there we're just going to do all traffic for now then we click save and there we are so now we have two security groups db servers and web servers a network acl is a network access control list and it's an optional layer of security for your vpc that acts as a firewall for controlling traffic in and out of one or more of your subnets you might set up network acls with rules similar to your security groups in order to add an additional layer of security to your vpc here is our network diagram and we've added network acls to the mix now you can see they sit somewhere between the root tables and the subnets this diagram makes it a little bit clearer and you can see that a network acl sits in between a root table and a subnet and also you can see an example of the default network acl which is configured to allow all traffic to flow in and out of the subnets to which is associated each network acl includes a whose rule number is an asterisk this rule ensures that if a packet doesn't match any of the other numbered rules it's denied you can't modify or remove this rule so if you take a look at this table you can see on the inbound some traffic would come in and it would look for the first rule which is 100 and that's saying i'm allowing all traffic from all sources so that's fine the traffic comes in if that rule 100 wasn't there it would go to the asterix rule and the aztecs rule is saying traffic from all sources is denied let's take a look at the network acl rules each subnet in your vpc must be associated with an acl if you don't assign it to a custom acl it will automatically be associated to your default acl a subnet can only be associated with one acl however an acl can be associated with multiple subnets an acl contains a list of numbered rules which are evaluated in order starting with the lowest as soon as a rule matches traffic it's applied regardless of any higher numbered rules that may contradict it aws recommends incrementing your rules by a factor of 100 so there's plenty of room to implement new rules at a later date unlike security groups acls are stateless responses to allowed inbound traffic are subject to the rules for outbound traffic welcome to the network acl demonstration where we're just going to have an overview of ocls where they are in the dashboard now you don't need to know a huge amount about them for the exam you just need to know how they work and where they are so let's go to networking and vpc and on when the dashboard loads on the left hand side under security there's network acls so let's click on that now you can see some acls that are in my my aws account so we want the one that's associated with our simply learn vpc so if we extend this vpc column that's our network acl simply then vpc now let's give it a name because it's not very clear to see otherwise also i'm kind of an obsessive tagger so let's call it simply learn acl and click on the tick so there we are so now it's much easier to see so we click on inbound rules so this is exactly what we showed you in the lesson the rule is 100 so that's the first rule that's going to get evaluated and it's saying allow all traffic from all sources and the outbound rules are the same so if you wanted to tighten down the new rule you could click edit we would give it a new rule number say which would be 200 so you should always increment them in 100 so that means if you had 99 more rules you needed to put in place you'd have space to put them in in between these two and then you could do whatever you wanted you could say you know we are allowing http access from all traffic and we're allowing or you could say actually you know what we're going to deny it so this is the way of blacklisting traffic into your vpc now i'm not going to save that because we don't need it but this is where network acls sit and this is where you would make any changes it's also worth having a look at the subnet associations with your acl so we have two subnets in our simply learn vpc so we would expect to see both of them associated with this network acl because it's the default and there they are this both are public and our private subnets are associated and you can also see up here on the on the dashboard it says default so this is telling us this is our default acl if you did want to create a new network acl you would click create network acl you'd give it a name just say new acl and then you would associate it with your vpc so we would say simply learn dpc takes a few seconds and there we are there we have our new one now you can see this one says default no because it obviously isn't the default acl for our simply learn vpc and it has no subnets associated with it so let's just delete that because we don't need it but there you are there's a very brief overview of network acls welcome to the amazon vpc best practices and costs where we're going to take a look at the best practices and the costs associated with the amazon virtual private cloud always use public and private subnets you should use private subnets to secure resources that don't need to be available to the internet such as database services to provide secure internet access to the instances that reside in your private subnets you should provide a nap device when using that devices you should use a nat gateway over nat instances because they're a managed service and require less administration effort you should choose your cidr blocks carefully amazon vpc can contain from 16 to 65 536 ip addresses so you should choose your cidr block according to how many instances you think you'll need you should also create separate amazon vpcs for development staging test and production or create one amazon vpc with separate subnets with a subnet each for production development staging and test you should understand the amazon vpc limits there are various limitations on the vpc components for example you're allowed 5 vpcs per region 200 subnets per vpc 200 root tables per vpc 500 security groups per vpc 50 in and outbound rules per vpc however some of these rules can be increased by raising a ticket with aws support you should use security groups and network acls to secure the traffic coming in and out of your vpc amazon advises to use security groups for white listing traffic and network acls for blacklisting traffic amazon recommends tiering your security groups you should create different security groups for different tiers of your infrastructure architecture inside vpc if you have web tiers and db tiers you should create different security groups for each of them creating tier-wise security groups will increase the infrastructure security inside the amazon vpc so if you launch all your web servers in the web server security group that means they'll automatically all have http and https open conversely the database security group will have sql server ports already open you should also standardize your security group naming conventions following a security group naming convention allows amazon vpc operation and management for large-scale deployments to become much easier always span your amazon vpc across multiple subnets in multiple availability zones inside a region this helps in architecting high availability inside your vpc if you choose to create a hardware vpn connection to your vpc using virtual private gateway you are charged for each vpn connection hour that your vpn connection is provisioned and available each partial vpn connection hour consumed is billed as a full hour you'll also incur standard aws data transfer charges for all data transferred via the vpn connection if you choose to create a nat gateway in your vpc you are charged for each nat gateway hour that your nat gateway is provisioned and available data processing charges apply for each gigabyte processed through the net gateway each partial nat gateway hour consumed is billed as a full hour this is the practice assignment for designing a custom vpc where you'll create a custom vpc using the concepts learnt in this lesson using the concepts learnt in this lesson recreate the custom vpc as shown in the demonstrations the vpc name should be simply learn vpc the cidr block should be 10.0.0.0.16 there should be two subnets one public with a range of 10.0.1.0 and one private with a range of 10.0.2.0 and they should be placed in separate availability zones there should be one internet gateway and one that gateway and also one custom route table for the public subnet also create two security groups simply learn web server security group and simply learn db server security group so let's review the key takeaways from this lesson amazon virtual private cloud or vpc enables you to launch aws resources into a virtual network that you've defined this virtual network closely resembles a traditional network that you'd operate in your own data center but with the benefits of using scalable infrastructure of aws there are three types of ip address in aws a private ip address this is an ip address that's not reachable over the internet and it's used for communication between instances in the same network a public ip address is reachable from the internet which you can use for communication between your instances and the internet and there's an elastic ip address this is a static public persistent ip address that persists after an instance restarts whereas a public ip address is raised associated after each restart amazon defines a subnet as a range of ip addresses in your bpc you can launch aws resources into a subnet that you select and a subnet is always mapped to a single availability zone use a public subnet for resources that must be connected to the internet and a private subnet for resources that won't be connected to the internet to allow your vpc the ability to connect to the internet you need to attach an internet gateway to it and you can only attach one internet gateway per vpc a root table determines where network traffic is directed it does this by defining a set of rules every subnet has to be associated with a root table and a subnet can only have an association with one root table however multiple subnets can be associated to the same root table and you can use a nat device to enable instances in a private subnet to connect to the internet or other aws services but and that device will prevent the internet from initiating connections with instances inside your private subnet a security group acts as a virtual firewall that controls the traffic for one or more instances you add rules to each security group that allow traffic to or from its associated instances a network access control list or network acl is an optional layer of security for your vpc that acts as a firewall for controlling traffic in and out of one or more of your subnets today's session is on aws sagemaker let's look into what we have in our today's session so what's in it for you we would be covering what is aws why do we need aws sage maker what is aws sage maker services what are the benefits of using the aws sagemaker machine learning with aws sagemaker how to train a model with aws stage sagemaker how to validate a model with aws and the companies that are using aws shmaker along with that we will be covering up one live demo on the aws platform now let's understand what is aws so what is aws it's an amazon web services it's a largest or most widely used public cloud platform offered by amazon it provides services over the internet aws services can be used to build monitor and deploy any type of application in the cloud aws also uses the subscription pricing model that means you only pay for whatever the services you use for now why do we need aws sagemaker let's look into it so let's consider an example of one of the company that is proquest now before aws sh maker the proquest is a global information content and technology company that provides valuable content such as ebooks newspapers etc to the users before awschmaker the proquest requirement was to have a better user experience maximum relevant search results now after aws sh maker they were able to achieve those results so they achieved more appealing media user experience they achieved more relevant search results for the users now what do we mean by aws age maker why this service is primarily used so amazon sage maker is a cloud machine learning platform that helps users in building training tuning and deploying machine learning models in a production ready hosted environment so it's kind of a machine learning service which is already hosted on the aws platform now what are the benefits of using aws sagemaker uh the key benefits of using aws sagemaker are it reduces machine learning data cost so you can do the cost optimization while running this particular service on the aws all ml components are stored in a particular place in a dashboard so they can be managed together highly scalable so it can be scalable on you can scale this particular service on the fly it trains the models quite faster maintains the uptime so you can be assured that your workloads will be running all the time it will be available all the time high data security so security becomes a major concern on the cloud platforms and it ensures that you have the high data security along with that can do a data transfer to different aws services like s3 bucket and all with these simple data transfer techniques now machine learning with aws sagemaker let us look into it so machine learning with aws sagemaker is a three step function so one is to build second is to test and tune the model and third is to deploy the model now with the build it provides more than 15 widely used ml algorithms for training purpose now to build a model you can collect and prepare training data or you can select from the amazon s3 bucket also choose and optimize the required algorithm so some of the algorithms that you can select are k means linear regressions logistic regression sagemaker helps developers to customize ml instances with the jupyter notebook interface in the test and tune you have to setup and manage the environment for training so you would need some sample data to train the model so train and tune a model with the amazon sagemaker sagemaker implements hyperparameter tuning by adding a suitable combination of algorithm parameters also it divides the training data and stores that in the amazon s3 s3 is a simple storage service which is primarily used for storing the objects and the data hence it is used for storing and recovering data over the internet and below you can see that aws sagemaker uses amazon s3 to store data as it's safe and secure also it divides the training data and stores in amazon s3 where the training algorithm code is stored in the ecr ecr stands for elastic container registry which is primarily used for containers and dockers ecr helps users to save monitor and deploy docker and the containers later maker sets up a cluster for the input data trains it and stores it in the amazon s3 itself so this is done by the sagemaker itself after that you need to deploy it so suppose you want to predict limited data at a time you use amazon sage maker hosting services for that okay but if you want to get prediction for an entire data set prefer using amazon sage maker batch transform now the last step that is to deploy the model so once tuning is done models can be deployed to sagemaker endpoints and in the end point real time prediction is performed so you would have some data which you would reserve and validate your model whether it is working correctly or not now evaluate your model and determine whether you have achieved your business goals now the other aspect is how we can train a model with aws stage maker so this is basically a flow diagram which shows you how to train a model with the aws sagemaker and here we have used couple of services of an aws to get that done so model training in aws age maker is done on machine learning compute instances and here we can see there are two machine learning compute instances used as helper code and the training code along with that we are using 2s3 buckets and the ecr for the container registry now let's look into what are the ways to train the model as per the slides so below are the following requirements to train a model so here in the diagram you can see these are the following requirements to train a model the url of an amazon s3 bucket where the training data is stored that is mentioned the compute resources on machine learning compute instances so these are all your machine learning compute instances then the url of an amazon s3 bucket where the output will be stored and the path of aws elastic container registry where the code data is safe the inference code image lies in the elastic container registry now what are these calls these are called as the training jobs now when a user trains a model in amazon sage maker he she creates a training job so we need to first create a training job and then the input data is fetched from the specified amazon s3 bucket once the training job is built amazon sage maker launches the ml compute instances so these compute instances will be launched once the training job is built then it trains the model with the training code and the data set and it shows the output and model rt crafts in the aws s3 bucket so this is done automatically now here the helper code performs a task when the training code fails the interference code which is in the elastic container registry consists of multiple linear sequence containers that process the request for inference on data the ec2 container registry is a container registry that helps users to save monitor and deploy container images whereas container images are the ready applications once the data is trained the output is stored in the specified amazon s3 bucket so here you can see the output will be stored here to prevent your algorithm being deleted save the data in amazon sagemaker critical system which can process you on your ml compute instances now how to validate a model let's look into it so you can evaluate your model using offline or using the historical data so first thing is that you can do the offline testing to validate a model you can do an online testing with the live data so if you have a live data coming or real time streams coming you can validate a model from there itself you can validate using a hold out set and also you can validate using the k-fold validation now use historical data to send requests to the model through the jupyter notebook in amazon sagemaker for the evaluation online testing with live data deploys multiple models into the endpoints of amazon sagemaker and directs live traffic to the model for validation validating using a holdout set is part of the data is set aside where which is called holdout set so the part of the data is left which is basically called as the holdout set this data is not used for the model training so later when the model is trained with the remaining input data and generalize the data based on what is learnt initially so whatever the data which is left out will be used for validating a model because we have not used the data while training a model the k-fold validation is the input data is split into two parts one part is called k which is the validation data for testing the model and the other part is k minus 1 which is used as a training data now based on the input data the machine learning model evaluates the final output now the companies that are using aws sagemaker one is the adp so you must be knowing about adb zelando dow jones which is on the stock market proquest and the intuit now let's look into the demo that how we can actually run the aws sagemaker so we'll use the r algorithm and then package the algorithm as a container for building training and deploying a model we are going to use the jupyter notebook for that for model building for model training for model deployment and the code for the demo is in the below link so you can see here that from this link you can get the code for the demo let's try to do a demo on the aws now i would be using a link which is provided by amazon to build train and deploy the machine learning model on the sage maker as you can see on my screen and in this tutorial you would have some steps where you can put those steps and the code python codes into your awschmaker jupiter lab so in this tutorial you will learn how to use amazon sagemaker to build train and deploy a machine learning model and for that we will use the popular xt boost ml algorithm for this exercise so first of all what you need to do is you have to go to the aws console and there you have to create a notebook instance so in this tutorial you will be creating a notebook instance you will prepare the data train the model to learn from the data deploy the model evaluate your ml model's performance and once all those activities are done then we will see how we can actually remove all the resources in order to prevent the extra costing now the first step is we have to enter to the amazon sage maker console so here you can see i'm already logged in into the sagemaker console you can click on the services search for the sagemaker here and here you get the amazon sagemaker service now the next step is that we have to create a notebook instance so we will select the notebook instance from the sagemaker service and then after the notebook instance is selected we'll put a name to our instance and we'll create a new im role for that so let's wait for the sagemaker studio to open so here you can see the studio is open and you just have to click on the notebook instances and here you have to create a notebook instance so here you can see couple of notebook instances have already been created one of them is in service so this is the notebook instance that we are going to use for creating the demo model i'll show you how you can create a notebook instance you just have to click on create notebook instance button and put your notebook instance name so you can put something like demo dash sage maker 987 or we can put it as model we'll go with notebook instance type as default which is mlt2.medium and in the permission and encryptions under the im rule we will click on create a new im role now why we are creating a new im rule so that we can allow the sage maker to access any s3 bucket that has been created on our account just click on create a role and here you would see that the new im role will be created with the set of permissions then rest of the things will keep it as default and then you just have to click on create a notebook instance the notebook instance creation takes some time so you just have to wait for a couple of minutes to get that in service we already have one of the notebook instance that has been created so we will be using that to create a demo now going back to the steps so these are the steps that we have already performed now once the notebook instance is created then we have to prepare the data so in this step we will be using the sagemaker notebook instance to pre-process the data that would require to train the machine learning model and for that we would be opening up the jupiter notebook and then we have to select an environment a kernel environment in the jupyter notebook that would be conda underscore python 3. so let's follow these steps go back to the sage maker click on the notebook instances select the running notebook instance and here you would select the open jupiter lab now here you would see that the sagemaker would try to open up the jupiter notepad and we would be performing all our inputs into that jupiter notebook and executing the results there itself so just wait for the notebook to open now here you can see the jupiter lab notebook has been open so i would be selecting one of the notebook that has been created so this one so likewise you can create your own notebook also how you can do that first of all let me select the kernel environment so i would be selecting corner underscore python 3 and just click on select so how you can create your own notebook just have to click on file click on new and here you can select the notebook just name your notebook select the environment conda underscore python 3 to run this demo so i have my notebook open so in the tabs i would be putting up the python codes and i would be executing those codes to get the output directly so the next step is to prepare the data train the ml model and deploy it we will need to import some libraries and define a few environment variables in the jupyter notebook environment so i would be copying this code which you can see that would try to import numpy pandas these are all required to run the python syntax so just copy this code and paste it into your notebook right so once you do that execute your code and here you can see that you get the output which is that it has imported all the necessary libraries that have been defined in the code now the next step is we would create an s3 bucket into the s3 service and for that you have to copy this python code just that you have to edit it so you have to specify this bucket name that you want to get created so here i would provide the bucket name which should be unique should not overlap so something like sage maker dash demo is the name that i have selected for the bucket and now you have to execute that code it says that the s3 bucket has been created successfully with the name sage maker dash demo 9876 so this is something which you can verify so you can go to the s3 service and there you can verify whether the bucket has been created or not now the next task is that we need to download the data to the aws sagemaker instance and load it into the data frame and for that we have to follow this url so from this url which is built train deploy machine learning model would have a data in the form of bank underscore clean.csv and this will be deployed onto our sagemaker instance we'll copy this code and paste it here and execute the code so it says that it has successfully downloaded bank underscore clean.csv which is which has the data inside it and that has to be loaded into the sagemaker data frame successfully now we have a data to build and train our machine learning model so what we are going to do we are going to shuffle the data and we are going to split it one into the training data set and the other one into test data set so for the training data set we are going to use 70 of the customers that are listed in the csv file and 30 of the customers in the csv file data we will be using it as a test data to train the model so we'll copy the following code into a new code cell and then we are going to run that code cell so i'll just copy it for training the data so that we can segregate the data model 70 for building the model and 30 for testing the data so click on run the execution and here you can see that we got the output successfully now we have to train the model from that data so how we are going to train that model and for that we will use sage maker pre-built xg boost model which is an algorithm so you will need to reformat the header and first column of the training data and load the data from the s3 bucket so what i'll do is i'll copy this syntax and paste it in the note cell so it has the trained data it would train the model click on run execution now it is changing the s3 input class which will be renamed to training input because now we are training the model with the training data so we just have to wait for some time till it gets executed completely now the next thing is that we need to set up the amazon sage maker session to create an instance of the xgboost model so here we are going to create this sagemaker session say we are going to create an instance of the xt boost model which is an estimator so just copy that copy that code and paste it here execute it and here you can see that it will start it has basically changed the parameter image name to the image underscore uri in this sagemaker python sdk v2 now we will follow the next step that is with the data loaded in the xt boost estimator we will set up train the model using gradient optimization and we'll copy the following code and that would actually start the training of the model so copy this code and this would actually start training the model using our input data that we have reserved 70 of that data that we have reserved for training the model so just copy that again initiate the execution and it will start the training job now we'll deploy the model and for that i would copy the deploy code put that in the cell and execute it so it says parameter image will be renamed to image uri and using already existing model so hd boost was deployed already uh if you have not done that if you're doing it the first time so it will initiate another xd boost instance so where you can find your xt boost endpoints created you just have to scroll down and here under the inference click on the end points and you should find the xg boost endpoints defined here so here you can see that today i have created one xt boost endpoint and that is now in process of creating so just refresh it so it is still created it is going to take some time to get that in service now our endpoint is in service state so now we can use it so going forward with the next steps we'll try to predict whether the customer in the test data enroll for the bank product or not for that we are going to copy this code put that in the jupyter cell function and execute it so here it gives you the output that it has actually evaluated and the same output we got in the screenshot of the demo as well now we are going to evaluate the model performance so what we are going to do we are going to get the prediction done so based on the prediction we can conclude that you predicted a customer that will enroll for a certificate of deposit accurately for 90 of the customers in the test data with the precision of 65 percent for enrolled and 90 which are which haven't enrolled for it so for that we are going to copy this code and executed here in the cell so if it is predicted correctly that means our model is working absolutely fine so here you can say the overall classification rate is 89.5 percent and there is the accurate prediction that has been made by the model and that's what the output we can see here in the screenshot of a model so that means our model is absolutely working fine it has been built deployed and trained correctly now the next thing is that once you are done with that you terminate your resources and for that you just have to copy this code and put that in the cell function so that the additional resources and the endpoints and the buckets that have been created by the jupiter notepad should be terminated so that you would not be incurred with the extra costing so just execute it and here you would see that it has tried to it would try to terminate all the additional resources that we have created from the jupiter today's tutorial is on aws cloudfront let's look into what we have today in the cloud front so what's in it for you we would be covering up the concept of what is aws what was earlier uh before aws cloudfront after aws cloudfront what were the services that were introduced how it benefited what do we mean by aws cloudfront benefits of the using aws cloudfront and how aws cloudfront actually is known as a content delivery service the name of the companies that are using aws cloudfront and we would be covering up one live demo now aws is the amazon web services it's a cloud service provider that basically offers a multiple services variety of services such as compute power database storage networking and other resources so that you can create your solutions on the cloud and help the business grow now with aws you only pay for whatever the services you use so for example if you are using a service for a couple of hours then you pay for only that many hours that you have used that service before aws cloud front so there is an example that we are going to talk and that is you must be aware of an application called spotify so when you used to access spotify and when you click on it it kept on loading and at the end you used to get the error and the error was that the connection failed and why you received that error because of a latency issue probably a network error right so how you can solve these kind of a latency issues and that is also going to these kind of an issues are also going to impact the performance of an application so with the introduction of aws cloud from this problem of loading the application got resolved so after aws cloud front with the help of idle base cloud front spotify gives the facility of updating new features access to million songs that you can access instantly so with the use of aws cloud front or the latency issues were solved and successfully you can basically access your application now what we mean by aws cloudfront so aws cloudfront is a globally distributed network that is offered by aws amazon web services which securely delivers content to the end users across any geography with a higher transfer speed and an improve or a low latency now what are the benefits of aws cloudfront there are multiple benefits one is the cost effective so it helps you to do the cost optimization when you use the cloud front it is time saving so it is implemented easily and also a lot of issues with respect to accessing the application with respect to latency and all can be resolved content privacy so the content is placed to the end users and also to the cloudfront servers in a secured manner in a secured way it is highly programmable and you can make the changes amend the changes on the fly and you can target any location any geography across the globe along with that it helps you to get the content deliver quickly now how aws cloudfront delivers the content let's look into the architecture so this is a flow and the flow is with respect to how the user is going to get a content from the cloud front now the client first accesses a website by typing a url on the browser and in the step one it acts tries to access the application then the client requests when the website is open the client request for an object to download such as for example a particular file now at that time the dns routes user request to download that file to aws cloudfront the aws cloud front connects to the nearest edge locations edge locations are basically the servers where it caches the files documents and the web codes aws cloud front connects to its nearest edge location in order to serve the user the request at edge location aws crowdfund looks for its requested cache file once the file is found let's say if the file is available in the cache of an edge location aws cloudfront then sends the file to the user otherwise if the file is not found in the cache memory of an edge location ews cloud front compares the requirement with the specification and share it with a respected server that means a web server or a server where the file is actually available the server the web server responds to the edge location by sending the file back to the cloud front edge location and then as soon as the aws cloud front receives the file it shares with the client also adds the file to the cache of an edge location for a future reference this is how the flow of a cloud front is now the name of the companies that are using the aws cloud front so one of them is jio7 app which is which is a very popular app so it uses amazon cloudfront to deliver 15 petabytes of audio and video to its subscribers globally which is a huge data sky news it uses the service in order to unify the content for faster distribution to scrub subscribers or the discovery communications also uses the cloud front it uses the service for delivering api static asset and also the dynamic content and then the tv one eu streaming europe is basically also uses the cloudfront service that helps in improving latency and performance that results in fastest delivery of content now let's look into the demo how to use cloudfront to serve private s3 bucket as a website now i'm going to run a cloudfront distribution demo on the aws console and we'll basically try to deliver the content from a private s3 bucket and then map that with the domain name using the root 53 service so what we need for this demo we would need a domain url we would need a route 53 service we would need cloud front uh we have to create a cloud front distribution and that will be linked with our s3 bucket the private s3 bucket right and in the s3 bucket we would have one html file the index.html file so let's uh move into aws console so right now i have opened up the cloudfront distribution and here you can see that couple of distributions have already been created so what i'm going to do i'm going to create a distribution now there are two types of delivery method for your content one is the web distribution and the other one is rtmp rtmp stands for real time audio or video distribution it's basically used for distribution of a media content or a media file which are available in the s3 bucket here we are going to select a web distribution because primarily we will be using files which uses protocols http or the https so you have to click on get started and in the origin domain name you have to specify the bucket where your code is available so i have a bucket already created here you can see and what you need to do is you have to create a bucket with the url name or the domain name which you would be mapping with the route 53 service so this is a bucket that has already been created let me show you in a new tab so here you have to go to the storage under the storage you have to select the s3 bucket let's open the link in the new tab and let's look into how we can create the s3 buckets now here are a couple of buckets already created i have created one bucket with the domain name that i'm going to use and map it with the route 53 service so that is basically mapped to a region which is in ohio and if you open up this bucket here you will find an html webpage the index.html has been already added right so similarly you have to create a bucket with a domain and an index.html page needs to be uploaded there now again we'll go to the cloud front we will try to create a distribution just have to click on create distribution select a web delivery method select an origin domain which is sunshinelearning.n and origin path you don't have to specify origin id is this one so basically when you define an origin domain name automatically the origin id appears you can customize this origin id as per your requirement also so rest of the things primarily we keep it as a default settings only until and unless if we require some customized settings to be done so let's say if you have to change the cache behavior settings you can do that otherwise we'll keep it as default now in the distribution setting you can either select use all edge locations for the best performance so what does aws basically do here it uses all the edge locations which are associated with aws across the globe otherwise you can specify based on the regions also right apart from that if you want to enable firewalls or the access control list you can specify here and then what you need to do is in the default root object here you have to specify your index.html page which is in the s3 bucket the distribution state has to be enabled and if you want to use ipv6 as well you need to enable it click on create distribution now you you can see here a distribution has been created it's in progress and it is enabled and it takes around 15 to 20 minutes to get that distribution completed the reason is that the web codes the web pages will be distributed across all the edge locations across the globe so hence it takes time to get that done right now let's move on to root 53 service and let's create the hosted zones so we'll type root 53 here scalable dns and domain name registration and what we are going to do here is we are going to map our url the domains pointed to [Music] the name servers that will be provided by the route 53 so we have to create a hosted zone let's wait for it so now the route 53 dashboard is open and you can see one hosted zone is already created so just click on the hosted zones and in order to point the traffic from the external domains towards the aws you have to first point the domain's traffic to the hosted zone in the route 53. so i'll click on create hosted zone but before that i will first delete the existing one and then i'll create another record another hosted zone right put your domain name let's say i put as sunshine learning dot in and it is acting as a public hosted zone rest of the things will be default click on create hosted zone now it gives you four name servers and these four name servers has to be updated in the domain so you have to update these name servers in a platform from where you have purchased the domain right so this is half of the work done then what you need to do is you have to go and create records now in the records you can select a routing policy so right now what we are targeting we are targeting basically that the traffic from the domain should be pointed directly towards the cloudfront distribution hence we are going with a simple routing right now click on next here you have to specify the record sets so we are going to create the records uh just click on default simple record put here as world wide web and you have to select an end point so end point we are selecting for the cloudfront distribution so we have to specify alias for the cloudfront distribution now here we are going to put the cloudfront distribution url and then we are going to define the simple report set so what we need is we need a cloudfront distribution url which you can find it uh from the cloudfront service itself and you can find the domain name here itself you just have to copy that and then paste it here in the distribution and then just click on define simple records again click on create records and here you can see the record set has been updated now this domain is basically pointed towards these name servers which are further pointed towards the cloud front distribution right now the only thing which is left is that within the domain from wherever you have purchased the domain you should update these phone name servers and then you can see the live traffic coming on this domain will have an output from the cloud front distribution today's topic is on aws auto scaling so this is akil i would be taking up a tutorial on the auto scaling let's begin with our tutorial and let's look into what we have in today's session so i would be covering up why we require aws auto scaling what is aws auto scaling what are the benefits of using the scaling service how this auto scaling works the different scaling plans we have what is the difference between the snapshot and the ami what is the load balancer and how many types of load balancers we have and along with that i would be covering up a real life demo on the aws let's begin with why we require aws auto scaling now before aws cloud scaling there was a question in the mind of enterprises that they were spending a lot of money on the purchase of the infrastructure if they have to set up some kind of a solution so they have to purchase an infrastructure and one time cost was required so that was a burden for them in terms of procuring a server hardware software and then having a team of experts to manage all those infrastructure so they used to think that no longer they require these resources if there was a cost efficient solution for their project that was the project manager used to think now after the aws cloud scaling that was introduced automatically the auto scaling maintains the application performance based on the user requirements at the lowest possible price so what does the auto scaling does is that whenever there is a scalability required it manages it automatically and hence the cost optimization became possible now what is aws auto scaling let's look into deep so aws auto scaling is a service that helps users to monitor their applications and the servers and automatically adjust the capacity of their infrastructure to maintain the steadiness so they can increase the capacity they can even decrease the capacity also for the cost optimization and also predictable performance at the lowest possible cost now what are the benefits of auto scaling it gave the better fault tolerance applications you can get the servers created and you can have a clone copy of the servers so that you don't have to deploy the applications again and again better cost management because the scalability is decided by the aws automatically based on some threshold parameters it was a reliable service and whenever the scaling is created or initiated you can get the notifications onto your mail ids or to your cell phones scalability as i mentioned is always there in the auto scaling it can scale up it can scale down as well and it has the flexibility the flexibility in terms of whenever you want to schedule it if you want to stop it if you want to keep the size of the servers at a fixed number uh you can always make the changes on the fly and the better availability now with the use of the auto scaling we come around with the terminology called snapshot and the ami let's look into the difference between the snapshots and the ami snapshots versus ami so in a company there was one of the employee that was facing an issue with launching the virtual machines so he asked his colleague a question is it possible to launch multiple virtual machines with a minimum amount of time because it takes a lot of time in terms of creating the virtual machines the other colleague said that yes it is possible to launch multiple ec2 instance and that can be done at a lesser time and with the same configuration and this can be done either you use a snapshot or the ami on the aws then the colleague said that what are the differences between the snapshot and ami let's look into the difference now the snapshots basically kind of a backup of a single abs volume which is just like a virtual hard drive that is attached to the ec2 instance whereas the ami it is basically used as a backup of an ec2 instance only the snapshots opts for this when the instance contain multiple static ebs when you opt for the snapshot whenever the instance contains multiple static evs volumes ami this is widely used to replace the failed ec2 instance in the snapshots here you pay only for the storage of the modified data whereas with the ami you pay only for the storage that you use the snapshots are non-bootable images on ebs volume whereas ami are bootable images on the ec2 instance however creating an ami image will also create the evs snapshots now how does aws auto scaling work let's look into it so for the aws auto scaling to work you have to configure single unified scaling policy for application resource and this scaling policy with that you can explore the applications also and then select the service you want to scale also for the optimization select do you want to optimize cost or do you want to optimize the performance and then keep track of scaling by monitoring or getting the notifications now what are the different scaling plans we have so in the auto scaling a scaling plan basically helps a user to configure a set of instructions for scaling based on the particular software requirement the scaling strategy basically guides the service of aws auto scaling on how to optimize resources in a particular application so it's basically a kind of the parameters that you set it up so that how the resource optimization can be achieved in the auto scaling with the scaling strategies users can create their own strategy based on their required metrics and thresholds and this can be changed on the fly as well what are the two types of scaling policies we have so there are basically dynamic scaling and the predictive scaling now what is dynamic scaling it basically guides the service of aws auto scaling on how to optimize the resources and it is helpful in optimizing resources for availability and particular price now with scaling strategies users can create their plan based on the required metrics and thresholds so a metric can be like let's say a network in network out or it can be a cpu utilization memory utilization likewise now in the predictive scaling its objective is to predict future workload based on daily and weekly trends and regular forecast future network traffic so it is kind of a forecast that happens based on the previous past experiences it uses a machine learning technique for analyzing that network graphic and this scaling is like how weather forecast works right it provides schedule scaling actions to ensure the resource capacity is available for application requirement now with the auto scaling you would need the load balancers also because if there are multiple instances that are created then you would need a load balancer to distribute the load to those instances so let's understand what do we mean by a load balancer a load balancer basically acts as a reverse proxy and it is responsible for distributing the network or the application traffic across multiple servers with the help of a load balancer you can achieve a reliability you can achieve a fault tolerance of an application that is basically it increases the fault tolerance and the reliability so for example when there is a high network traffic that is coming to your application and if that much traffic comes to your application to the instances your instances may crash so how you can avoid that situation so you need to manage the network traffic that is coming to your instances and that can be done with the load balancer so thanks to the aws load balancers which helps in distributing network traffic across backend servers in a way that it increases performance of an application here in the image you can see the traffic coming from a different resources landing on to the ec2 instance and the load balancer is actually distributing that traffic to all the three instances hence managing the network traffic quite properly now what are the types of load balancers we have there are three types of load balancers on the aws one is the classic load balancer second is the application load balancer and the third one is the network load balancer let's look into what we have in the classic load manager so the classic load balancer is the most basic form of load balancing and we call it as a primitive load balancer also and it is widely used for the ec2 instances it is based on the ip address and the tcp port and it routes network traffic between end users as well as in between the backend servers and it does not support host-based routing and it results in low efficiency of resources let's look into what we have in the application load balancer this is one of the advanced forms of load balancing it performs a task on the application level in the osi model it is used when there are http and https traffic routing is required and also it supports the host-based and path-based routing and performs well with the services or the backend applications the network load balancer performs the task at layer 4 of the connection level in the osi model the prime role of the network load balancer is to route the tcp traffic and it can manage a massive amount of traffic and is also suitable to manage the low latencies let's look into the demo and see how practically we can create the auto scale guys let's look into the demo for how we can create an auto scaling on the aws console so right now i'm logged in into the aws console and i am in the mumbai region what you need to do is you have to go to the compute section and under that click on the ec2 service let's wait for the ec2 servers to come now just scroll down and under the load balancing there is an option called auto scaling so there first you have to create a launch configuration and then after that you have to create the auto scaling groups so click on launch configuration and then you have to click on create launch configurations so click on create launch configuration now this launch configuration is basically this set of parameters that you define while launching and auto scaling so that this uniformity is maintained with all the instances so that includes let's say if you select a windows os or a linux os that particular type of an operating system will be implemented in all the instances that will be part of an auto scaling so there are certain set of parameters that we have to specify during the launch configuration so that we can have a uniformity in terms of launching the servers so here i would select an amazon linux ami and then i would select the type of a server which will be t2.micro click on configure details put the name to the launch configuration let's say we put it as a demo and the rest of the things we'll keep it default click on add storage since it's a linux ami we can go with the 8gb storage that should be fine click on configure security group let's create a new security group which has the ssh port opened and that is open for anywhere which is basically source ipv4 and ipv6 ips any ip will be able to access that click on review just review your launch configuration if you want to make changes you can do that otherwise click on create a launch configuration you would need the key pair and this key pair will be a unique key pair which will be used with all the instances that are part of the auto scaling group so we can select an existing key pair if you have that otherwise you can create a new key pair so i have an existing key pair i'll go with that acknowledge it and click on create launch condition now we have successfully launched the configuration of an auto scaling the next thing is to create an auto scaling group so click on create an auto scaling group using this launch configuration put a group name let's say we put something like test and the group size to start with uh it says one instance so that means at least a single instance will always be running and it will be initiated and running 24 cross 7 till the auto scaling is available you can increase the size of the minimum base instances also let's say you can change it to 2 also so you would get at least two servers running all the time so we'll go with the one instance the network would be the vpc default and uh in the vpc particular region we can select the availability zones so let's say if i select availability zone 1a and then availability zone 1b so how the instances will be launched so one instance will be launched in one and the other one in the one b the third one in the one a fourth one and the one b likewise it will be equally spreaded among the vulnerability zones next part is to configure the scaling policies so click on it if you want to keep this group at its initial size let's say if you want to go with only a single instance or two instances and you don't want the scaling to progress you can put it keep this group at its initial size so this is basically a way to hold the scaling but we will use the scaling policies to adjust the capacity of this group so click on it and we would scale between let's say minimum one instance that we have and we'll scale it between one two four instances and what condition on what basis these instances will be scaled up or scaled down would be defined in the scale group size so the scaling policies you can implement based on a scale group size or using the simple scaling policies using the steps so in the scale group size you have certain metrics you can use average cpu utilization you can define a metric related to average networking average network out or the load balancer request counts per target and if you create the simple scaling policies using steps then you need to create the alarms and there you can get some more metrics that you can add up as a parameter for the auto scaling let's go with the scaling group size let us go with the metric type as average cpu utilization and the target value here you have to specify what would be the threshold that when the instance cpu utilization is crossed then a new instance should be initiated so you can put a reasonable threshold for that let's say we put something like 85 percent and whenever the instance cpu utilization is crossed 85 threshold you will see that there will be a new instance created let's go to the next configure notifications and here you can add notifications so let's say if there is a new instance that is initiated and you want to basically be notified so you can get notifications over your email ids or you can get it on the cell phones so for that for adding the notification you would need the sns service that is called as a simple notification service and you have to create a topic there you have to subscribe for the topic using your email id and then you should get the notifications click on configure tags the tags are not mandatory you can basically put a tag let's say if you want to identify the instance for the purpose it was created otherwise you can leave it blank also click on review and review your scaling policies notification tags as well as the scaling group details click on create auto scaling group and here you go your scaling has been launched click on close and you should get at least a single instance initiated automatically by the auto scaling so let's wait for the details to appear so here you can see our launch configuration name demo auto scaling group name test minimum instance we want one the maximum instances we want four we have selected two availability zones ap south one ap south 1b and the instance one has been initiated and if you want to verify where exactly this instance has been initiated just click on the instances here and here you will see that our single instance has been initiated that is in service and that has been initiated in ap south 1b now once the threshold of this instance crosses 85 percent that is what we have defined in the scaling policies then you should see that another instance will be initiated so likewise this is basically uh i have created steps to initiate a scaling policy that means to increase the number of servers whenever the threshold crosses likewise here itself you can add another policy to scale down the resources in case if the cpu utilization goes to a normal value today we are going to discuss about amazon redshift which is one of the data warehouse service on the aws so let's begin with amazon redshift and let's see what we have for today's session so what's in it for you today we'll see what is aws uh why we require amazon redshift what do we mean by amazon redshift the advantages of amazon redshift the architecture of amazon redshift some of the additional concepts associated with the redshift and the companies that are using the amazon redshift and finally uh we'll cover up one demo which will show you the practical example that how you can actually use the redshift service now what is aws as we know that aws stands for amazon web service it's one of the largest cloud providers in the market and it's basically a secure cloud service platform provided from the amazon also on the aws you can create and deploy the applications using the aws service along with that you can access the services provided by the aws over the public network that is over the internet they are accessible plus you pay only whatever the service you use for now let's understand why we require amazon redshift so earlier before amazon redshift what used to happen that the people used to or the developers used to fetch the data from the data warehouse so data warehouse is basically a terminology which is basically represents the collection of the data so a repository where the data is stored is generally called as a data warehouse now fetching data from the data warehouse was a complicated task because might be a possibility that the developer is located at a different geography and the data data warehouse is at a different location and probably there is not that much network connectivity or some networking challenges internet connectivity challenges security challenges might be and a lot of maintenance was required uh to manage uh the data warehouses so what are the cons of the traditional data warehouse services it was time consuming to download or get the data from the data warehouse maintenance cost was high and there was the possibility of loss of information in between the downloading of the data and the data rigidity was an issue now how these problems could overcome and this was uh basically solved with the introduction of amazon redshift over the cloud platform now we say that amazon redshift has solved traditional data warehouse problems that the developers were facing but how what is amazon redshift actually is so what is amazon redshift it is one of the services over the aws amazon web services which is called as a data warehouse service so amazon redshift is a cloud-based service or a data warehouse service that is primarily used for collecting and storing the large chunk of data so it also helps you to get or extract the data analyze the data using some of the bi tools so business intelligence tools you can use and get the data from the redshift and process that and hence it simplifies the process of handling the large scale data sets so this is the symbol for the amazon redshift over the aws now let's discuss about one of the use case so dna is basically a telecommunication company and they were facing an issue with managing their website and also the amazon s3 data which led down to slow process of their applications now how could they overcome this problem let's say that so they overcome this issue by using the amazon redshift and all the company noticed that there was 52 increase in the application performance now did you know that amazon redshift is basically cost less to operate than any other cloud data warehouse service available on the cloud computing platforms and also the performance of an amazon redshift is the fastest data warehouse we can say that that is available as of now so in both cases one is that it saves the cost as compared to the traditional data warehouses and also the performance of this red shift service or a data warehouse service the fastest available on the cloud platforms and more than 15 000 customers primarily presently they are using the amazon redshift service now let's understand some of the advantages of amazon redshift first of all as we saw that it is one of the fastest available data warehouse service so it has the high performance second is it is a low cost service so you can have a large scale of data warehouse or the databases combined in a data warehouse at a very low cost so whatever you use you pay for that only scalability now in case if you wanted to increase the nodes of the databases in your redshift you can actually increase that based on your requirement and that is on the fly so you don't have to wait for the procurement of any kind of a hardware or the infrastructure it is whenever you require you can scale up or scale down the resources so this scalability is again one of the advantage of the amazon redshift availability since it's available across multiple availability zones so it makes this service as a highly available service security so whenever you create whenever you access redshift you create a clusters in the redshift and the clusters are created in the you can define a specific virtual private cloud for your cluster and you can create your own security groups and attach it to your cluster so you can design the security parameters based on your requirement and you can get your data warehouse or the data items in a secured place flexibility and you can remove the clusters you can create under clusters if you are deleting a cluster you can take a snapshot of it and you can move those snapshots to different regions so that much flexibility is available on the aws for the service and the other advantage is that it is basically a very simple way to do a database migration so if you're planning that you wanted to migrate your databases from the traditional data center over the cloud on the redshift it is basically uh very simple to do a database migration you can have some of the inbuilt tools available on the aws access you can connect them with your traditional data center and get that data migrated directly to the redshift now let's understand the architecture of the amazon redshift so architecture of an amazon redshift is basically it combines of a cluster and that we call it as a data warehouse cluster in this picture you can see that this is a data warehouse cluster and this is a representation of a amazon redshift so it has some of the compute nodes which does the data processing and a leader node which gives the instructions to these compute nodes and also the leader node basically manages the client applications that require the data from the redshift so let's understand about the components of the redshift the client application of amazon redshift basically interact with the leader node using jdbc or the odbc now what is jdbc it's a java database connectivity and the odbc stands for open database connectivity the amazon redshift service using a jdbc connector can monitor the connections from the other client applications so the leader node can actually have a check on the client applications using the jdbc connections whereas the odbc allows a leader node to have a direct interaction or to have a live interaction with the amazon redshift so odbc allows a user to interact with live data of amazon redshift so it has a direct connectivity direct access of the applications as well as the leader node can get the information from the compute nodes now what are these compute nodes these are basically kind of a databases which does the processing so amazon redshift has a set of computing resources which we call it as a nodes and the nodes when they are combined together they are called it as a clusters now a cluster a set of computing resources which are called as nodes and this gathers into a group which we call it as a data warehouse cluster so you can have a compute node starting from 1 to n number of nodes and that's why we call that the redshift is a scalable service because we can scale up the compute nodes whenever we require now the data warehouse cluster or the each cluster has one or more databases in the form of a nodes now what is a leader node this node basically manages the interaction between the client application and the compute node so it acts as a bridge between the client application and the compute nodes also it analyzes and develop designs in order to carry out any kind of a database operations so leader node basically sends out the instructions to the compute nodes basically perform or execute that instruction and give that output to the leader node so that is what we are going to see in the next slide that the leader node runs the program and assign the code to individual compute nodes and the compute nodes execute the program and share the result back to the leader node for the final aggregation and then it is delivered to the client application for analytics or whatever the client application is created for so compute nodes are basically categorized into slices and each node slice is alerted with specific memory space or you can say a storage space where the data is processed these node slices works in parallel in order to finish their work and hence when we talk about a redshift as a fast faster processing capability as compared to other data warehouses or traditional data warehouses this is because that these node sizes work in a parallel operation that makes it more faster now the additional concept associated with amazon redshift is there are two additional concepts associated with the redshift one is called as the column storage and the other one is called as the compression let's see what is the column storage as the name suggests column storage is basically kind of a data storage in the form of a column so that whenever we run a query it becomes easier to pull out the data from the columns so column storage is an essential factor in optimizing query performance and resulting in quicker output so one of the examples are mentioned here so below example show how database tables store record into disk block by row so here you can see that if we wanted to pull out some kind of an information based on the city address age we can basically create a filter and from there we can put out the details that we require and that is going to fetch out the details based on the column storage so that makes data more structured more streamlined and it becomes very easier to run a query and get that output now the compression is basically to save the column storage we can use a compression as an attribute so compression is a column level operation which decreases the storage requirement and hence it improves the query performance and this is one of the syntax for the column compression now the companies that are using amazon redshift one is lya the other one is equinox the third one is the pfizer which is one of the famous pharmaceuticals company mcdonald's one of the burger chains across the globe and philips it's an electronic company so these are one of the biggest companies that are basically relying and they're putting their data on the redshift data warehouse service now in another video we'll see the demo for using the amazon redshift let's uh look into the amazon redshift demo so these are the steps that we need to follow for creating the amazon redshift cluster and in this demo what we will be doing is that we will be creating an im rule for the redshift so that the redshift can call the services and specifically we'll be using the s3 service so the role that we'll be creating will be giving the permission to redshift to have an access of an s3 in the read-only format so in the step one what we require we'll check the prerequisites and what you need to have is the aws credentials if you don't have that you need to create your own credentials and you can use your credit and the debit card and then in the step 2 we'll proceed with the im role for the amazon redshift once the role is created we'll launch a sample amazon redshift cluster mentioned in the step 3 and then we'll assign a vpc security groups to our cluster now you can create it in the default vpc also you can create a default security groups also otherwise you can customize the security groups based on your requirement now to connect to the sample cluster you need to run the queries and you can connect to your cluster and run queries on the aws management console query editor which you will find it in the redshift only or if you use the query editor you don't have to download and set up a sql client application separately and in the step 6 what you can do is you can copy the data from the s3 and upload that in the redshift because the redshift would have an access a read-only access for the s3 as that will be created in the im role so let's see how we can actually use the redshift on the aws so i am already logged in into my account i am in north virginia region i'll search for redshift service and here i find amazon redshift so just click on it let's wait for the redshift to come now this is a redshift dashboard and from here itself you have to run the cluster so to launch a cluster you just have to click on this launch cluster and once the cluster is created and if you wanted to run queries you can open query editor or you can basically create queries and access the data from the red shift so that's what it was mentioned in the steps also that you don't require a separate sql client application to get the queries run on the data warehouse now before creating a cluster we need to create the role so what we'll do is we'll click on the services and we'll move to i am role section so i am role i can find here under the security identity and compliance so just click on the identity access management and then click on create roles so let's wait for i am page to open so here in the im dashboard you just have to click on the rules i already have the role created so what i'll do is i'll delete this role and i'll create it separately so just click on create role and under the aws services you have to select for the redshift because now the redshift will be calling the other services and that's why we are creating the role now which other services that the redshift will be having an access of s3 why because we'll be putting up the data on the s3 and that is something which needs to be uploaded on the redshift so we'll just search for the redshift service and we can find it here so just click on it and then click on red shift customizable in the use case now click on next permissions and here in the permissions give the access to this role assign the permissions to this role in the form of an s3 read-only access so you can search here for the s3 also let's wait for the policies to come in here it is let's type s3 and here we can find amazon s3 read-only access so just click on it and assign the permissions to this role tags you can leave them blank click on next review put a name to your role let's put my redshift role and click on a create role now you can see that your role has been created now the next step is that we'll move to redchat service and we'll create one cluster so click on the services click on amazon redshift you can find that in the history section since we browsed it just now and from here we are going to create a sample cluster now to launch a cluster you just have to click on launch this cluster whatever the uncompressed data size you want in the form of a gigabyte terabyte or petabyte you can select that and let's say if you select in the form of gb how much db memory you want you can define it here itself this also gives you the information about the costing on demand is basically pay as you use so they are going to charge you 0.5 dollars per r for using the two node slices so let's click on launch this cluster and this will be a dc2 dot large kind of an instance that will be given to you it would be in the form of a solid state drive ssds which is one of the fastest way of storing the information and the nodes two are mentioned by default that means there will be two node slices and that will be created in a cluster you can increase them also let's say if i put three node slices so it is going to give us 3 into 0.16 db per node storage now here you have to define the master username password for your redshift cluster and you have to follow the password instructions so i would put a password to this cluster and if it accepts that means it does not give you any kind of a warning otherwise it is going to tell you about you have to use the ascii characters and all and here you have to assign this cluster the role that we created recently so in the available im rules you just have to click on my redshift role and then you have to launch the cluster if you wanted to change something in with respect to the default settings let's say if you wanted to change the vpc from default vpc to your custom vpc and you wanted to change the default security groups to your own security groups so you can switch to advanced settings and do that modification now let's launch the cluster and here you can see the redshift cluster is being created now if you wanted to run queries on this redshift cluster so you don't require a separate sql client you just have to follow the simple steps to run a query editor and the query editor you will find it on the dashboard so let's click on the cluster and here you would see that the red shift cluster would be created with the three nodes in the us east 1b availability zone so we have created the redshift cluster in the ohio region and now what we'll do is we'll see how we can create the tables inside the redshift and we'll see how we can use the copy command so that we can directly move the data uploaded on the s3 bucket to the redshift database tables and then we'll query the results of a table as well so how we can do that first of all after creating the redshift cluster we have to install the sql workbench slash j this is not a mysql which is managed by oracle and you can find this on the google you can download it from there and then you have to connect this client with the redshift database how you can do click on file click on connect window and after connecting a window uh you have to paste the url which is a jdbc driver this driver link you can find it onto the aws console so if you open up a redshift cluster there you would find the jdbc driver link let's wait for it so this is our cluster created let's open it and here you can find this jdbc url and also make sure that in the security groups of a redshift you have the port 5439 open for the traffic incoming traffic you also need to have the amazon redshift driver and this is the link where you can download the driver and specify the path once you are done with that you provide the username and the password that you created while creating the redshift cluster click on ok so this connects with the database and now the database connection is almost completed now what we will be doing in the sql workbench we'll be first creating the sales table and then in the sales table we'll be adding up the entries copied from the s3 bucket and then move it to the redshift database and after that we'll query the results in the sales table now whatever the values you are creating in the table the same values needs to be in the data file and i have taken up this sample data file from this link which is docs.aws.amazon.com redshift sample database creation and here you can find a download file ticketdb.zip file this folder has basically multiple data files sample data files which you can actually use it to practice uploading the data on the redshift cluster so i have extracted one of the files from this folder and then i have uploaded that file in the s3 bucket now we'll move into the s3 bucket let's look for the file that has been uploaded on the s3 bucket so this is the bucket sample and sales underscore tab dot text is the file that i have uploaded this has the entries data entries that will be uploaded using a copy command onto the red shift cluster now after executing after putting up the command for creating up the table then we'll use a copy command and copy command we have to define the table name the table name is sales and we have to define the path from where the data would be copied over to the sales table in the red shift now path is the s3 bucket and this is the redshift bucket sample and it has to look for the data inside the sales underscore tab.text file also we have to define the role arn that was created previously and once it is done then the third step is to query the results inside the sales table to check whether our data has been uploaded correctly on the table or not now what we'll do is we'll execute all these three syntax it gives us the error because we have to connect it again to the database let's wait for it execute it it's again gives us the error let's look into the name of the bucket it's redshift bucket sample so we have two t's mentioned here right let's connect with the database again and now execute it so table sales created and we got the error the specified bucket does not exist uh redshift bucket sample let's view the bucket name redshift bucket sample let's copy that put it here connect to the window connect back to the database right and now execute it so table sales created uh the data in the table has been copied from the s3 bucket to sales underscore tab.text to the redshift and then the query of the results now the results from the table has been queried hi guys i'm raul from simply learn and today i'd like to welcome you all to the greatest debate of the century today i'm joined by two giants of the cloud computing industry they'll be going head to head with each other to decide who amongst them is better it's going to be one hello fight now let's meet our candidates on my left we have aws who's voiced by a picture hi guys and on my right we have microsoft azure who's voiced by anjali hey there so today we'll be deciding who's better on the basis of their origin and the features they provide their performance on the present day and comparing them on the basis of pricing market sharing options free tier and instance configuration now let's listen to their opening statements let's start with aws launched in 2006 aws is one of the most commonly used cloud computing platforms across the world companies like adobe netflix airbnb htc pinterest and spotify have put their faith in aws for their proper functioning it also dominates the cloud computing domain with almost 40 percent of the entire market share so far nobody's even gotten close to beating that number aws also provides a wide range of services that covers a great number of domains domains like compute networking storage migration and so much more now let's see what azure has to say about that azure was launched in 2010 and is trusted by almost 80 percent of all fortune 500 companies the best of the best companies in the world choose to work only with azure azure also provides its services to more regions than any other cloud service provider in the world azure covers 42 regions already and 12 more are being planned to be made azure also provides more than 100 services spanning a variety of domains now that the opening statements are done let's have a look at the current market status of each of our competitors this is the performance route here we have the stats for the market share of aws azure and other cloud service providers this is for the early 2018 period amazon web services takes up a whopping 40 of the market share closely followed by sgr at 30 and other cloud services adding 30 this 40 indicates most organizations clear interest in using aws we are number one because of our years of experience and trust we've created among our users sure you're the market leader but we are not very far behind let me remind you more than 80 percent of the fortune 500 companies trust azure with their cloud computing needs so it's only a match of time before azure takes the lead the rest of the 30 person that is in aws or azure accounts to the other cloud service providers like google cloud platform backspace ibm software and so on now for our next round the comparison route first we'll be comparing pricing we'll be looking at the cost of a very basic instance which is a virtual machine of two virtual cpus and 8gbs of ram for aws this will cost you approximately 0.0928 us dollars per hour and for the same instance in azure it will cost you approximately 0.096 us dollars per hour next up let's compare market share and options as i mentioned before aws is the undisputed market leader when it comes to the cloud computing domain taking up 40 percent of the market share by 2020 aws is also expected to produce twice its current revenue which comes close to 44 billion dollars not to mention aws is constantly expanding its already strong roadster of more than 100 services to fulfill the shifting business requirements of organizations all that is great really good for you but the research company gardner has released a magic quadrant that you have to see you see the competition is now neck-to-neck between azure and aws it's only a matter of time before azure can increase from its 30 market share and surpass aws this becomes more likely considering how all companies are migrating from aws to azure to help satisfy their business needs azure is not far behind aws when this comes to services as well azure service offerings are constantly updated and improved on to help users satisfy their cloud computing requirement now let's compare aws and azure's free offerings aws provides a significant number of services for free helping users get hands-on experience with the platform products and services the free tier services fall under two categories services that will remain free forever and the others that are valid only for one year the always free category offers more than 20 services for example amazon sms sqs cloudwatch etc and the valid fourier category offers approximately 20 services for example amazon s3 ec2 elastic cache etc both types of services have limits on the usage for example storage number of requests compute time etc but users are only charged for using services that fall under the valid for a year category after a year of their usage a shop provides a free tier as well it also provides services that belong to the categories of free for a year and always free there are about 25 plus always free services provided by azure these include app service functions container service active directory and lots more and as of the valid for a year there are eight services offered there's linux or windows virtual machines glob storage sql database and few more azure also provides the users with credits of 200 us dollars to access all their services for 30 days now this is a unique feature that azure provides where users can use their credits to utilize any service of a choice for the entire month now let's compare instance configuration the largest instance that aws offers is that of a whopping 256 gb of ram and 16 virtual cpus the largest that azure offers isn't very far behind either 224 gbs of ram and 16 virtual cpus and now for the final round now each of our contestants will be shown facts and they have to give explanations for these facts we call it the rapid fire round first we have features in which aws is good and azure is better aws does not cut down on the features it offers its users however it requires slightly more management on the user's part azure goes slightly deeper with the services that fall under certain categories like platform as a service and infrastructure as a service next we have hybrid cloud where aws is good and azure is better okay although aws did not emphasize on hybrid cloud earlier they are focusing more on technology now azure has always emphasized on hybrid cloud and has features supporting it since the days of its inception for developers aws is better and azure is good of course it's better because aws supports integration with third-party applications well azure provides access to data centers that provide a scalable architecture for pricing both aws and azure are at the same level it's good for aws because it provides a competitive and constantly decreasing pricing model and in the case of azure it provides offers that are constantly experimented upon to provide its users with the best experience and that's it our contestants have finished giving their statements now let's see who won surprisingly nobody each cloud computing platform has its own pros and cons choosing the right one is based entirely on your organization's requirements hi guys today we've got something very special in store for you we're going to talk about the best cloud computing platform available amazon web services uh rahul i think you said something wrong here the best cloud computing platform is obviously google cloud platform no it isn't aws has more than 100 services that span a variety of domains all right but google cloud platform has cheaper instances what do you have to say about that well i guess there's only one place we can actually discuss this a boxing ring so guys i'm apexah and i will be google cloud platform and i'm rahul i'll aws so welcome to fight night this is aws versus gcp the winner will be chosen on the basis of their origin and the features they provide their performance in the present day and comparing them on the basis of pricing market share and options the things they give you for free and instance configuration now first let's talk about aws aws was launched in 2004 and is a cloud service platform that helps businesses grow and scale by providing them services in a number of different domains these domains include compute database storage migration networking and so on a very important aspect about aws is its years of experience now aws has been in the market a lot longer than any other cloud service platform which means they know how businesses work and how they can contribute to the business growing also aws has over 5.1 billion dollars of revenue in the last quarter this is a clear indication of how much faith and trust people have in aws they occupy more than 40 of the market which is a significant chunk of the cloud computing market they have at least 100 services that are available at the moment which means just about every issue that you have can be solved with an aws service now that was great but now can we talk about gcp i hope you know that gcp was launched very recently in 2011 and it is already helping businesses significantly with a suite of intelligent secure and flexible cloud computing services it lets you build deploy and scale applications websites services on the same infrastructure as google the intuitive user experience that gcp provides with dashboards wizards is way better in all the aspects tcp has just stepped in the market and it is already offering a modest number of services and the number is rapidly increasing and the cost for a cpu instance or regional storage that gcp provides is a whole lot cheaper and you also get a multi-regional cloud storage now what do you have to say on that i'm so glad you asked let's look at present day in fact let's look at the cloud market share of the fourth quarter of 2017. this will tell you once and for all that aws is the leader when it comes to cloud computing amazon web services contributes 47 of the market share others like rackspace or verizon cloud contribute 36 microsoft azure contributes 10 the google cloud platform contributes 4 and ibm software contributes 3 47 of the market share is contributed by aws you need me to convince you any wait wait wait all that is fine but we only started a few years back and have already grown so much in such a less span of time haven't you heard the latest news our revenue is already a billion dollars per quarter wait for a few more years and the world shall see and aws makes 5.3 billion dollars per quarter it's going to take a good long time before you can even get close to us yes yes we'll see now let's compare a few things for starters let's compare prices for aws a compute instance of two cpus in 8gb ram costs approximately 68 us dollars now a computer instance is a virtual machine in which you can specify what operating system ram or storage you want to have for cloud storage it costs 2.3 cents per gb per month with aws you really want to do that because gcp wins this hands down let's take the same compute instance of two cpus with 8gb ram it will cost approximately 50 dollars per month with gcp and as for my calculations that's a 25 percent annual cost reduction when compared to aws talking about cloud storage costs it is only 2 cents per gb per month with gcp what else do you want me to say let's talk about market share and options now aws is the current market leader when it comes to cloud computing now as you remember we contribute at least 47 percent of the entire market share aws also has at least 100 services available at the moment which is a clear indication of how well aws understands businesses and helps them grow yeah that's true but you should also know that gcp is steadily growing we have over 60 services that are up and running as you can see here and a lot more to come it's only a matter of time when we will have as many services as you do many companies have already started adopting gcp as a cloud service provider now let's talk about things you get for free with aws you get access to almost all the services for an entire year with usage limits now these limits include an hourly or by the minute basis for example with amazon ec2 you get 750 hours per month you also have limits on the number of requests to services for example with aws lambda you have 1 million requests per month now after these limits are crossed you get charged standard rates with gcp you get access to all cloud platform products like firebase the google maps api and so on you also get 300 in credit to spend over a 12 month period on all the cloud platform products and interestingly after the free trial ends you won't be charged unless you manually upgrade to a paid account now there is also the always free version for which you will need an upgraded billing account here you get to use a small instance for free and 5gb of cloud storage any usage above this always free usage limits will be automatically billed at standard rates now let's talk about how you can configure instances with aws the largest instance that's offered the 128 cpus and 40 views of ram now other than the on demand method like i mentioned before you can also use spot instances now these are the situations where your application is more fault tolerant and can handle an interruption now you pay for the spot price which is effective at a particular r now these part prices do fluctuate but are adjusted over a period of time the largest instance offered with google cloud is 160 cpus and 3.75 tbs ram like spot instances of aws google cloud offers short lived compute instances suitable for bad jobs and fault tolerant workloads they are called as preemptable instances so these instances are available at eighty percent off on on demand price hence they reduce your compute engine costs significantly and unlike aws these come at a fixed price google cloud platform is a lot more flexible when it comes to instance configuration you simply choose your cpu and ram combination of course you can even create your own instance types this way before we wrap it up let's compare on some other things as well telemetry it's a process of automatically collecting periodic measurements from remote devices for example gps gcp is obviously better because they have superior telemetry tools which help in analyzing services and providing more opportunities for improvement when it comes to application support aws is obviously better since they have years of experience under their bed aws provides the best support that can be given to the customers containers are better with gcp a container is a virtual process running in user space as kubernetes was originally developed by google gcp has full native support for the tools other cloud services are just fine tuning a way to provide kubernetes as a service also the containers help with abstracting applications from their environment they originally run it the applications can be deployed easily regardless of their environment when it comes to geographies aws is better since they have a head start of a few years aws in this span of time has been able to cover a larger market share and geographical locations now it's time for the big decision so who's it going to be yeah who is it going to be gcp or aws i think i'm going to go for in cloud computing the competition for leadership is a quite tough three-way race amazon web services microsoft azure and google cloud platform clearly are the top cloud companies who are commanding lead in the infrastructure as a service and platform as service markets hey everyone i'm shamlee and i welcome you all to this new video of simply learn on aws versus azure versus gzp in this video we will compare and contrast aws azure and gcp based on a few related concepts around these cloud computing platforms it will help us understand the functioning of these top cloud platforms and will also let us figure out the individuality of each one of them but before starting with the comparison let's have a quick introduction of aws versus assured versus gcp so let's get started amazon web services or aws is a cloud computing platform that manages and maintains hardware and infrastructure reducing the expense and complexity of purchasing and running resources on site for businesses and individuals these resources are available for free or for a fee per usage microsoft azure is a cloud computing service that it offers a collection of cloud computing services for building testing deploying and managing application in the cloud including remotely hosted and managed versions of microsoft technology google cloud platform offers a variety of cloud computing services for building deploying scaling monitoring and operating a cloud the services are identical to those that power google products such as google search gmail youtube and google drive now let's move on to the comparison between aws assure and gcp we will be comparing them based on few major parameters like origin service integration availability zone cloud tools like compute storage networking market share pricing and at last who uses them now let's move ahead and start with the first comparison origin in the year 2006 amazon web services aws was introduced to the market and in the year 2010 azure launched its services whereas on the other hand gcp was established in the year 2008. from the start aws has been supportive of the open source concept but the open source community has a tense relationship with azure on the other hand gcp similar to aws provides google cloud with manage open source services that are tightly linked aws offers services on a large and complex scale that can be manipulated but azure support is comparatively low quality whereas gcp's monthly support price is almost 150 dollars for the silver class which is the most basic of services and is quite expensive now let's move on to the service integration of these cloud platforms service integration is a set of tools and technology that connects different applications systems repositories and data and process interchange in real time aws makes it simple for users to combine services such as amazon ec2 amazon s3 beanstalk and others on the other hand azure allows customers to effortlessly combine azure vms as your app service sql databases and other services whereas users can utilize gcp to combine services such as compute engine cloud storage and cloud sql now that we know briefly about all these cloud platforms let's have a look at the availability zones of these platforms because aws was the first in the cloud domain they have had more time to build and extend their network but azure and gcp both have various locations around the world but the distinction is in the amount of availability zones they have aws now offers 66 availability zones with an additional 12 on the pipeline close to it azure is available in 140 countries and is available in 54 regions throughout the world but google cloud platform is now available in 20 global areas with three more on the way now let's move on to the next important factor which is tools now let's move ahead and have have a look at the first feature which is compute elastic compute cloud or ec2 is aws compute service which offers a wide range of features including a large number of instances support for both windows and linux high performance computing and more as sure on the other hand virtual machines is microsoft azure's core cloud-based compute solution it includes linux windows server and other operating systems as well as better security and microsoft program integration in comparison to its competitors google's computing services catalog is somewhat smaller compute engine that companies principal service offers custom and predefined machine types per second invoicing linux and windows support and carbon neutral infrastructure that uses half the energy of traditional data centers within the compute category amazon's different containers services are gaining prominence it has docker kubernetes and it's also far gate service which automates server and cluster management when using containers as well as other alternatives azure unlike aws uses virtual machine scale sets of two container services azure container service is based on kubernetes and container service uses docker hub and azure container registry for management for enterprises interested in deploying containers google offers the kubernetes engine it's also worth noting that google was significantly involved in the kubernetes project providing it extensive knowledge in this field now let's move on to the next parameter of comparison which is storage simple storage service for object storage elastic block storage for persistent block storage and elastic file system for file storage among aws storage offerings block storage for rest based object storage of unstructured data queue storage for a large volume workloads file storage and disk storage are among microsoft azure's core storage services gcp offers an increasing number of storage options its unified object storage service cloud storage also has a persistent disk option relational database service or rds dynamodb nosql database elastic cache in memory data store redshift data warehouse neptune graph database and database migration service are all sql compatible databases offered by amazon the database choices in azure are specially wide sql database mysql database and postgre sql database are the three sql based choices when it comes to databases gcp offers the sql based cloud sql and cloud spanner a relational database built for mission critical workloads now the next parameter is networking aws uses amazon virtual private cloud or vpc on the other hand azure uses azure virtual network we net and gcp uses cloud virtual network now let's move on to the another factor which is market share and pricing all these cloud services are based on comparative pricing strategy which means you need to pay on the basis of its usage according to canalis the worldwide cloud market rose 35 to 41.8 billion in the first quarter of 2021. aws accounts for 32 of the market with azure accounting for 19 and google accounting for 7 on one hand amazon charges on a yearly basis and on the other hand microsoft azure and google services charge on a minute basis and also of them provide you a standard price for you to access these services aws charges roughly 69 per month for a very basic instance with two virtual cpus and 8 gb of ram and aws largest instance with 3.84 db of ram and 128 vcpus will set you back roughly 3.97 per hour but in azure the same type of instance one with two v cpus and 8gb of ram cost roughly 70 us per month and asia's largest instance has 3.89 tb of ram and 128 virtual cpus it costs about 6.79 per hour compared to aws gcp will supply you with the most basic instance which includes two virtual cpus and eight gigabytes of ram for 25 percent less as a result it will set you back roughly to 52 dollar every month and the largest instance that include is 3.75 db of ram and 160v cpus it will cost you around 5.32 per hour amazon other than this also provides spot instances reserved instances and dedicated hosts where you can look for multiple offers and discounts but for ensure it provides special prices to developers based on situations or even ensure hybrid benefit which benefits your organization up to 40 percent if it uses microsoft software in their data centers whereas google offers quite a sorted pricing to its customer compared to other two it gives you sustained use discounts which activate if you use the same instance for a month the printable instance which is very similar to amazon spot instances but one thing is common in all three cloud services which is they all offer long-term discount now let's have a look at the last comparison which is companies using them because aws is the oldest player in the cloud business it has the largest user base and community support as a result aws has a largest number of high-profile and well-known clients including netflix airbnb unilever bmw samsung mi senga and others with time azure is getting a large number of high-profile customers azure currently boasts around 80 percent of fortune 500 forms as customers johnson controls polycom fujifilm hp honeywell apple and others are among its key clients google cloud on the other hand uses the same infrastructure as google search and youtube and as a result many high in enterprises trust google cloud hsbc paper 20th century fox bloomberg domino's and other among google clouds many clients hey guys this is chidanan from simply learn and i welcome you to this short tutorial and demo on kubernetes kubernetes specific to aws cloud platform so what's a part of this tutorial and demo what's in store for you at offset i would like to cover the basics of orchestration tools as most of you would know kubernetes is one of the most popular orchestration tools in recent times specifically for applications which are cloud native and deployed on some or the other types of containers but what are these organization tools why would one need to use orchestration tools what are the facilities or features provided by this orchestration tools that's the first thing that i'm going to cover after that i will pick two orchestration tools specific to container management docker swarm versus kubernetes i'm going to compare them with regard to the features that they provide the use cases that they cover what facilities that they provide and when to use what after that i will get into a little bit details of the kuvurnis architecture what exactly is required for setting up a kubernetes cluster what runs in a control plane or a master node what runs as a part of the worker node after that i will end the tutorial by running you by setting up a three node kubernetes cluster on aws platform i will use something called as cops which is one of the admin tools for setting up production grade kubernetes cluster so i will use this to set up a three note cluster on aws all right now that we set the context right let me get started so what are orchestration tools the application development and the lifecycle management from the time the application is developed connecting the source code to your continuous integration to testing them all along your development process and eventually moving it down to production managing your production servers the dependencies that your software has the requirement that it has in terms of the hardware the features of fault tolerance self-healing capabilities auto scaling all this has complicated over the last few years as a part of devops one thing that everyone is interested in is managing all this application dependency or lifecycle management using some or the other kind of a tool so that's where these orchestration tools have become very very popular so what kind of features that they provide is you'll have to just let them know what are the kinds of tool sets that is required what are the fault tolerance mechanisms that it has to be adopted to what kind of a self-healing capabilities that application will have to need and if at all there is any auto scaling features that is required if at all you can bake in all these specific parameters into your tool your organization tool becomes a one-stop shop for all your deployment and configuration management needs that's where this orchestration tools have become a very very popular and relevant specifically these days when people are more and more uh interested in adopting devops practices all right so having said that about orchestration tools let's concentrate on two of these orchestration tools specifically for container management docker swarm and kubernetes many of the applications that are written for these kind of containers are called or kind of fall into a category of cloud native where the application need not know much about the underlying infrastructure or the platform where these applications are running so these two along with apache missiles there are many other container management systems but i'm going to pick docker swarm and kubernetes for comparing the features that is provided by these two frameworks for the sake of comparison i picked up docker swarm and kubuntis just because of the reason that both of them operate in the space of container management they do something very very similar to containers that's the similarity that exists between these two orchestration tools however there's a big difference that exists when it comes to the types of containers that both of them cater to and also the capacity of workloads they can be used for docker swamp is a cluster management solution that is provided by docker container docker is one of the very very popular containers of recent times and in case you have your applications that is totally powered by only docker containers if you have a need where you want to run a cluster of servers which are running only docker containers docker swarm should be your choice for your orchestration tool kubernetes on the other hand can cater to any other types of containers other than docker and including docker as well so in case you have your applications which have got docker in them which have got rkt in them which they have got alex in them or any other type of container kubernetes should be your choice of orchestration too a docker swarm can manage up to 40 50 or 60 max nodes so in case your application is totally written in docker containers and the load or the expected cluster size is about 50 60 nodes docker swarm should be your choice of orchestration tool on the other hand kubernetes is something that was open sourced by google and the kind of the scale of operations that kubernetes can cater to is google scale so if in case your applications are more than a thousand nodes that is where kubernetes comes into play that's the big difference that exists between docker swarm and kubernetes putting those differences aside let me compare docker swarm and kubernetes based upon other features which are similar in nature the first and important feature that kubernetes provides is something called as auto scaling if at all the load on your cluster is too high your application is experiencing more load so kubernetes can add new nodes to your cluster of course you'll have to configure kubernetes in order to have those capabilities where it can spin up new vms or new nodes if at all you do that configuration correctly kubernetes has got the capacity or it has got the feature where it can bring up a new node and add it to the cluster on the fly based upon the load that exist at that moment on similar lines if at all the load is not too high it can identify a few of those nodes which have got less number of replicas or less number of application containers running on it it can move them to some other node and also scale down by deleting few nodes from your cluster so that is the powerfulness of auto scaling which is provided by kubernetes and this unfortunately does not exist in docker swarm the other feature of load balancing specifically application node balancing so docker swarm gives you an application level auto load balancing however kubernetes gives you the flexibility of manually configuring any other type of load balancer of your choice for application load balancing installation as i mentioned earlier docker swarm is a loose cluster of containers which are running on nodes it is very easy to spin up a new node and then connect them to a swarm and this can be done in a very very loosely coupled way where you can create swarms of your choice notes are allowed to connect to swarms and quit or leave the swamps on the fly so in and all the installation is pretty easy and fast kubernetes on the other hand the configuration of kubernetes the way to spin up a big cluster specifying the size of the node how many masternodes how many config planes that you want how many nodes that you want it's a pretty tough thing to bring up a kubernetes scalability kubernetes strength is very very strong they are very tightly coupled and even they have the capability where on the cluster size things or the nodes can be increased on the fly based upon the requirement on a darkest form the cluster strength is weak as compared to cobra days worker notes can be added to a swarm worker notes can be asked to levers form or can be taken out of a swarm based upon the requirement so this is kind of a loosely coupled architecture for docker swarm while kubernetes the cluster is very tightly coupled since docker swarm runs only docker containers containers find it very easy to share data and lot of other things with each other because they all have the same signature they're all from the same family so it's very easy for them to share not just volumes but a lot of other things however for kubernetes since it manages containers of different types if your application has to share some data across different containers there's a little bit of a complexity in how you would want your containers to share data also when it comes to kubernetes kubernetes groups containers in something called as pods while a pod can have either one container that's the preferred choice or multiple containers and the idea of pod is like each part can run in any other node it's not guaranteed that you would want to have two or three parts to be running on the same note that makes data sharing a little bit of a different thing compared to docker's form uh gui so there's not a good enough ui tool in docker's form at least the ce edition of it which will allow you to get to know what is running on your containers what is the size of the containers what is the volume of the containers and all that stuff there are some free and open source tools like proteiner which gives you a good visibility into your running docker containers however there's nothing at a swarm level that is provided by docker kubernetes gives you an out of the box dashboard which is easy to configure and set up you can also club it with some metrics services and you can also get information about the size of the cluster that is running what is the load on the nodes and stuff like that all this across your cluster in a very beautiful dashboard perspective so that way um this is little bit of a good ui for kubernetes while for docker's form there isn't anything that is provided out of the box let me spend some time in explaining a little bit about the kubernetes architecture what comprises of the kubernetes cluster what resides in a master or a control plane and what resides in a worker node this is a very high level uh depiction of a kubernetes cluster so this is the master node which has got something called as a cluster store a controller a scheduler component and an api server and these are the bunch of nodes that are connected or administered by the master node the master node can be one three five typically an odd number this is as per the typical cluster management thing where you would need the masternodes to be in odd numbers so that whenever there's any contention in terms of what needs to be deployed where or where to give the job to whom and stuff like that all the masters would cast their vote and they would decide on the outcome so that's the master node and there are a bunch of nodes which can be connected and administered by the master node cuba ctl or the command from using which anybody can assign some workloads to the cluster can also be run either on the same node or on a separate node so cubectl is the command line tool that we will be installing and using in order to fire up our commands to our cluster so if at all i have to put up a job on our cluster if i have to give out any specific commands to my cluster all this is done using cuba ctl there's a bunch of rest apis which is used by cuba ctl and cubectl will talk to the api server and fire up the commands specific to my cluster what runs in the master node master node without saying it's the most important component of any of the cluster if at all you're running cluster with only one master node if your master node goes down there's no other way that you know you can or any user can talk to the different nodes in the cluster so master node is the most vital component responsible for the complete kubernetes cluster there's always one node that should be running as a master node that's the bare minimum requirement for your cluster so what are the other components in the master node the most important one is called etcd so this is nothing but a data store a value of key value pair which is stored which contains all information about your cluster so all the configuration details which node is up which worker node is down all this information is stored in the cluster store so all the managers would access this cluster store before they go ahead and decide any other work item or anything else that has to be done specific to your cluster what's in the controller as the name says controller is something it's like a daemon server that is running in a continuous loop all the time it is responsible for controlling the set of replicas the kind of workloads that is running based upon the configuration that the user has set in so if at all if you are aware of something called as replication controllers endpoint controllers namespace controllers all these controllers are managed by the controller component if any user asked for some three replicas of a particular pod or a service to be running and if at all any of the nodes goes down or the part goes down for whatever reason the controller is the one who wakes up and assigns this particular job or the part to some other available node by looking up for the details using in the cluster store that's the importance of the control manager or the controller the scheduler the scheduler assigns the tasks based upon whoever is asked for any job to be scheduled based upon a time frame or based upon some criteria it also tracks the working load as to what what exactly is the load who is running what and in the cluster and places the workload on whoever is the available resource at that time all right api server this is one other important component in in our kubernetes cluster where how would the end user deploy or give out any sort of a workload onto your cluster all the requests come to the api server so this is an entry point for all your requests that come into your cluster anybody wants to deploy something anybody want to scale up a controller anybody who wants to bring down a few services anybody wants to put in a service all this will have to come in as a part of a rest um api endpoint and api server is the enter point in case you don't want uh you know somebody to access your cluster in case you want only specific people or specific users to be running some specific workloads you can set all those role-based access control for this apa server so this is the entry point for anyone who wants to submit any job to your cluster so that's a quick overview about the master node what what are the important components of a master node now let's go over to what runs in a worker or a slave node as i mentioned earlier slave nodes are something where the job that is submitted to your cluster is eventually run as a pod as a service as a container so in kubernetes world there's something called as a pod a pod is nothing but a combination of a container it is a wrapper around your running containers so slave nodes or the worker nodes typically run these parts so that is where the whole workload typically gets run but there are a bunch of other components in the in the node which also manages what runs in the pod who has to have access to the pod what is the state of the pod is it in a running state is it going down for some reason and all the stuff so let's quickly go over those components that is there in the slave node the most important component in my opinion that should be on the on the node should be the container runtime as i mentioned earlier kubernetes can run any different types of containers not just docker so in case you want to have a node which wants to have docker running on it rkt running or lkc running on it or any other container environment that is running on it you have to ensure that the specific container runtime environment is available on that specific node so that whenever a job is submitted a description of what needs to be run as a part of the pod what should be the image with it should be powered and what kind of a container to spin up so that's what you know when the job is finally assigned to this particular node it would definitely need a container runtime to be up and running so that exists only if at all the container runtime is installed and running on your kubernetes worker node okay now that we know what our kubernetes node can run how would somebody assign a job to our node that is using something called as a cubelet as the name says cubelet or cubelet is a small subset which talks to the cube api server so any node which has to run any kind of a pod all these instructions are passed around by the qbpa server to the cubelet and cubelet is capable of processing whatever job that is assigned to it and ensuring that so many parts and so many services are spun up based upon the requirement the last component that exists in the worker node is the cube proxy or kubernetes proxy this plays a very very important role acting as a load balancer and a network proxy so what typically happens is whenever the pods are running in nodes the parts can be typically running in any node there is no um affinity towards any node on which these parts are running because pods or containers are something called as ephemeral they can run anywhere so how would somebody reach out to these applications that is running in some pod which is running in one container now and running in probably another container another node altogether tomorrow so that is where your kubernetes proxy contains a picture and this component will ensure that any container that is spun up or any pod that is spun up it keeps track of all these parts and kind of connects the end points or acts like a dns server so that it knows when somebody is trying to reach out to this particular service which is the part on which the service is typically running so that plays a very very important role by acting as a load balancer and a network proxy now that at a very high level we know what are all the components that make up of our kubernetes cluster let's go ahead and create a small cluster and spin up this cluster on aws platform finally we get to the demo section of my tutorial now that you guys are aware as to what is kubernetes what is the use of kubernetes you also know at a very very high level what are the components of a cluster what is a master node or a control plane what are the components of a master node and what should be existing in a worker node you're probably thinking that it's maybe pretty hard to kind of set this up so let me demystify that by running you with a quick demo of how to set up a simple pre-node kubernetes cluster using a tool called cops and i will be doing this whole setup on my aws cloud okay so what is cops cops is nothing but a simple tool this is an admin tool that allows you to bring up production grade kubernetes environments as i said setting up a kubernetes cluster on a bare metal is little challenging so that is why i would use something called as a cops and i will use this on my cluster which i'll spin up on my aws instance so i have an aws account and what i will do first is i will first spin up an ec2 server i will power this with one of the um linux amis and i will install cops on it now this will be my starting point from where i'm going to trigger running up a big cluster in our case i'll try to keep it little small i don't want too many nodes in my cluster i'm going to have one master node and two worker nodes this whole operation would be done programmatically by installing this cops admin tool so as you may know aws is little hard for me to programmatically run other components or rather bring up servers it's not pretty simple so what i would do is i will create an iam rule which will attach to my ec2 server so that my ec2 server gets powered with all the required role so that it can go ahead and spin up cluster based upon my configuration that i'm going to specify all right so once i have my cluster set up i will also install cuba ctl which uh you would probably know by now is nothing but a command line utility using which i can kind of connect to my cluster give it some jobs put some parts and stuff like that so i will use cuba ctl i will also have a pair of ssh keys so that from this machine i would be able to successfully get onto the master node and submit jobs on my behalf so let me begin by first logging into my aws account all the steps that i will be following to bring up my aws cluster kubernetes cluster will be documented and i'll be sharing you this document in a github repository so that in case anybody wants to try you'll be more than happy to find all the required instructions in one place okay now the first thing that i would need is to launch an aws ec2 instance now this is an instance where i'm going to install cops and i will use it as a base server from where i'm going to find some commands which will bring up my kubernetes cluster so let me login to my aws account and um let me stick to one particular region and let me bring up some one ec2 instance all right so let me launch my instance let me choose um doesn't matter in the ways but in case you want to try it on your free tire i would recommend choose free tire only and choose an ami of your choice let me choose this instance i will make it t2 micro that is good for me configuration instance details all right i don't want to change anything here that looks good maybe i will change this 30 gig that's good for me let me add a tag to my instance i will name this as my cops server all right i would need a bunch of posts to be opened up so what i will do is i will create a new security group uh called open to the word i would not recommend that you do this since i don't want to it will take a long time for me to specifically uh to pick and choose um the ports that has to be open for running my cluster i don't want to do that i will open up all the http and https ports so that it's quicker so i will say all tcp i would say from anywhere just to be on the safer side also open http and https and i will make this anywhere and anywhere this is good for me i will say review and launch all right so i chose a t2 micro instance just for the sake of easiness i've opened up all the ports and um the instant details are you've seen what is that i've chosen all right now important part is like i would need a keyboard i need to create a new keypad i will call this as simply learn keys i will download this keypair and then i will launch the instance it will take some time for my ec2 instance to come up in the meanwhile what i'll do is i will convert i have this pen file which is the pair of keys the ssh keys with which i need to log into my ec2 instance so i'll need to convert this because i'm trying to connect from my windows box i would need to convert this into a ppk file so i have puttygen i'm going to load my set of keys here and convert that key into a ppk key all right open all right i would say save hang on save private key yes i would say simply learn or simply learn private private key and save this here and that that's all so this is my public key which is the pem file and this is my tpk or my private key now using the private key i will log into my ec2 instance okay now my ec2 instance is up and running it is running in this ohio region and this is the ip address of my public ipads of my machine so let me connect to it so i use a moba x term which is nothing but an ssh emulator you can use putty or any of these sessions which allow you to do an ssh this is my ip address of my machine and since i've spun up an amazon ami the user this is the default user i would need to specify the private keys for my session and this is the private key for my session say okay i am in my amazon ec2 instance so let me look at what are the other things that i need to do all right so as i said i just brought up my ec2 instance but i would need my ec2 instance to run few things on my behalf so that it can spin up easy to other ec2 instances it can also talk to an s3 bucket where i'm going to store the instant state of my kubernetes cluster also some sort of an auto scaling group because i want to spin up more and more instances i also want to have a private hosted zone using my route 53 so i would need my ec2 instance to have all these sort of permissions for my server so what i would do is i will go and create a permission a rule in the ec2 or rather in the aws im role and i will attach this i am role to my ec2 instance all right so let me go to my iam which is nothing but the identity and access management i will create a role called as possibly corpse role i will create a role this role will be used by my ec2 so i'm going to click on ec2 and say next permissions they would need a lot of permissions uh specific permissions to be very honest and the permissions are all listed out here s3 ec2 and all this stuff just to keep it pretty simple what i would do is i'll create a role with the administrative access all right so i don't want any tags i will review i'll create a role name i will say cups fold for ec2 all right so i'm going to create a rule which has got administrative access rules i'm going to create a rule so this would ensure that my ec2 instance from which i'm going to run my cluster would have all the prerequisite permissions so that it can go and spin up instances talk to us three buckets and all that stuff all right now let us have our running instance get powered by this role so i will connect this role to my running instance all right so this is my easy to instant that is running action attach attach attach attach replace im rules so there is no rule as of now i would want my cops roll for easy this row that i created so i want this new role to be assigned to my ec2 all right great now my ec2 instance which is my cop server has got all the required permissions so that he can create a cluster on my behalf great now before this let me do a sudo yum update iphone y so that any of the because this is a newly provisioned vm i just want to ensure that all my library system libraries are all updated so that when i do install cop and cube ctl and all those things none of them will fail because of some dependency or package issues so i'm just running a sudo update iphone y so that all the libraries are updated okay looks like it is done so let me go ahead and install cops cups is a pretty simple installation it is available in a particular location i have to just copy this and ensure that i do a curl and i install this particular stuff all right so it is fetching uh the cops tool it's installing for me once it is copied down let me change the mode so that i can execute it and then also let me move it to my user local bin so that it is available for me in my path okay that's pretty much the corpse installation let me also do one other thing let me install something called as cuba ctl so this is what would be you know a tool using which i'm going to be firing my uh commands to my kubernetes cluster once it comes up right this is a pretty smaller executable so ctl all right so i have cuba ctl as well as cops installed on my ec2 server now okay now what i'll do next is i'm going to create an s3 bucket in aws so that the equipment is a cluster state is persisted in this bucket so let me create a simple bucket with this name s3 bucket with this name so let me go to s3 and let me create a simple bucket with this name i will just say create a bucket okay so this simply learn kubernetes is the bucket that i created so this bucket will store all information about my cluster let me now uh go and create a dns entry or a dns zone or aws route 53 hosted zone for my cluster so this is required so that you know i can give a cluster name for my kubernetes cluster that i'm coming up so i will create a private hosted zone and possibility possibly i will call this as simplylearn.n so this will be my name of my private hosted zone in case you already have any other public hosted zones in your name you can always put a route 53 hosted zone specific for your domain name since i don't have anyone i'm just going to create a simple private hosted zone so let me head down to route 53 click on any of these and you'll get these hosted zones out here so i'm going to create a hosted zone here i'm going to create a hosted zone click on create hosted zone um this would be simply learn lot in i want to create a public no i don't want public i want a private hosted zone and i'm going to associate my vpc id for this hosted zone since i'm going to try out all my exercises in the ohio region i will associate the vpc of the ohio region for this all right so this is the one that is specific to the ohio region so i will say this one i'll say create a hosted great now let me come back to my ec2 box my ec2 is all powered with whatever it needs in order to go ahead with the installation of my cluster only few things that i need to take care of is that you know now that i put a name for my cluster i also have an s3 bucket that i've configured for my cluster i will have to ensure that i need to put in these two configurations as a part of my cluster building activity so i will open my bash rc file and i'm going to export these two variables if at all you remember well these two variables are nothing but the cluster name which is nothing but the public sorry the private hosted zone that i created and the s3 bucket where i'm going to create or rather i'm going to store my cluster state so let me copy these two and open up my bashrc file all right so i will just add these two and copy these add export them out as my variable i'm going to save this here now let me ensure that this gets picked up all right i've got these two that have configured and i also ensure that these environment variables are set now let me create a pair of ssh keys this is to ensure that i can log into the box that i'm going to be provisioning as a part of my cluster ssh hyphen keygen i don't want to give any parameters let it go to the home directory with the default name and without any passphrase all right so i created a bunch of my keypair now i'm all set to go ahead and run my cops now cops will ensure that this will take some time for the cluster to come up but this is exactly how i would define my corpse command i have copied here so corpse create cluster the state of the cluster will be stored in this particular variable which is what i have exported out the number of node count is two node count is the worker node count if at all i don't specify configuration for specifying the master or the control pane as well if i don't specify that the default one is one so i'm going to create one primary or the master node and two uh worker nodes uh size of my master node size of the worker nodes uh what are the zones where i want this to be created and where do i store the cluster information and uh okay i've already added this master account this is actually not required but this is the command that i'm going to fire off so that i bring up my cluster all right that went off uh very fast but um this actually did not create a cluster it is just the definition of a cluster and that's why this came out very fast now that everything looks good with whatever configuration that i specified now let me go ahead and create the cluster by saying corpse update cluster iphone iphone 8 yes now this will take some time a good five to ten minutes for it to because this will actually start provisioning the servers and as i mentioned earlier based upon my configuration i'm gonna come up with one master server and uh two nodes or the worker nodes all right so this is the command for me to validate the cluster and i'm gonna try it out first i'm pretty sure that it will fail because all my servers are not up yet it is taking some time the validation everything failed but let me try to look at my ec2 instances and see how many servers do i have running as of now if you see i had only one server that i had started which was my cop server and automatically the other three instances one is called not start simply not in this these two are the nodes and this is the master node so these three got automatically provisioned by the cops command that i ran all right so this may take a while for it to get validated a good 5 to 10 minutes so what i'll do is i'll pause the video now and i'll come back once the server the cluster is up and running it's been a good uh eight to nine minutes uh since i started my cluster so let me validate now okay seems good so there's a master node minimum one max one some nodes which are nothing but the slave nodes or the worker nodes there are two of them due to micro subnet so my clusters seem to be up and running and my cluster name is simply learn.n so what i would want to do is now let me just log into the master and see if i can run some parts so let me get back to my installation steps here the validation cluster is done so let me log into my since my cluster name is simply non dot and this is the way to log into my box so let me get into my box so if you see here uh this is the host name of the machine that i'm currently in if you see this this is nothing but our this cop server this is a server now from here i'm trying to get into the master box all right so if you see the ip address has changed i was in a different box i'm in a different box now if i see host name you'll find a different host name so this is 153 which is nothing but the master node yep it's 153 this is the particular host so i'm getting into this machine now so i started the cluster from my cop server here it ran and brought up three nodes so i'm actually getting into my master node and see if i can run some parts on it all right so let me try kubectl uh get cluster info try cube ctl get nodes all right so there are three nodes here master node and one master node and two worker nodes so let me see if i have some parts here cubicle get pods so there's no port as of now so what i'll try to do is let me just pin up a very very simple part just to check if my connections is everything is correct or not correct i have a simple uh node.js application that i've built uh and i've got a container for that this is already pushed to the docker have registered is called simply learn docker hub and the image name that i have here is called my node app and i will use this image to power one of my parts that i will run so this is the way in which i'm going to launch my pod let me show you these commands cube ctl run the name for my deployment that i'm going to run hyphen iphone image and this is the image that is going to be powering my container so this is simply learn docker rub forward slash my node app hyphen f1 replicas equal to i want two replicas to be running and the port that i want to expose this particular part is on eight zero eight zero so let me run this as soon as this is run it creates something called as a deployment all right now let me just say um i'm not really interested in the deployment i'm interested in checking if all the parts are up and running so i will say cube ctl get both all right so that was pretty fast so i have two parts that are running here these are two replicas because i asked for two replicas these parts are running so i can run cube ctl describe pod pickup all right i can pick up any of the pod name and see what is the pod what is the information does it contain from what is the image that is pulling what is the image image name id this is actually running my pod which is actually spinning up a container great so far so good so in kubernetes the pods can are ephemeral so they can be there at any time cannot be read in time if i need to expose my pod outside i will have to create a service for my part so what i'll do is i'll create a very very simple service by exposing this particular deployment before that let me check cube cdl get deployment so there's a deployment that is created the deployment name is simply run app so i will expose my deployment simply learn app as uh yeah i'll just expose this let me see all right i am not specifying what type and all i don't want to get into the complications of uh what are the different types of exposing the service and all that stuff so if i don't specify anything uh it gives me something closer cluster ip so this is where my pod is actually exposed so let me just check if at all this part is up and running i'll just try a simple curl command curl http colon this is my cluster ip and the port is 8080. if at all i hit this it's actually hitting my application and giving me uh whatever i put a simple assist.out kind of a thing where i'm just printing um the container id of whatever pod is serving uh being sold out of so i'm actually hitting my container and i'm getting this output from the content so everything looks good my content is up and running uh so let me just go and clean it up i'll say equip ctl uh delete deployment simply learn app this should get rid of all the parts that i created cube ctl getpod all right these parts are intimidating things let me just check cubectl get services there's one service i don't want this service let me delete that cubectl delete service i want this all right that sounds good so let me come back to my host my cop server from where i'm running so i managed to successfully verify this part see if everything is up and running so what i'll do is i'll just go ahead and complete this demo by going and getting rid of my cluster so corpse delete cluster hyphen f1 yes all right so this will ensure that it will clean up my complete cluster that i created so i had three or four running instances if you see that all the three are shutting down because you know the cop server which had cops installed on it has now got a mandate to go ahead and shut down and clean up the whole instances and all those things that are created as a part of my deployment cloud computing is becoming more prominent among tech companies and organizations making this skill in demand in the sector cloud-based solutions are proven to be a boom for company management around the world because they provide convenience at an inexpensive price this means that if you want to work in technology a job dealing with cloud-based systems could be a great fit hey everyone i'm sharmly and i welcome you all to this new video of simply learn on how to become a cloud engineer according to market and markets the global cloud computing industry was valued at 371.4 billion usd in 2020. it has been predicted that cloud engineer will be among the top 10 in demand id jobs in 2021 therefore in this video we will come across some of the very important aspects needed to become a cloud engineer like first of all we will understand what is cloud computing then we will understand who is a cloud engineer after that we will look into the technical skills needed to become a cloud engineer after that we will understand what are the roles and responsibilities of a cloud engineer then we will have a look into the roadmap of becoming a cloud engineer and what is the average salary of a cloud engineer in both the usa and in india and at last we will look at the companies hiring cloud engineers if getting your learning started is half the battle what if you could do that for free visit skill up by simply learn click on the link in the description to know more the first thing we will come across is what is cloud computing cloud computing is the distribution of computing services via the internet or the cloud including servers storage databases networking software analytics and intelligence to provide faster innovation more flexible resources and economies of scale you usually only pay for the cloud services you use which helps you cut costs run your infrastructure more efficiently and scale as your business pros that brings us to who is a cloud engineer cloud engineers are the i.t experts in charge of accessing an organization's technology infrastructure and transferring operations and procedures to a cloud-based architecture they assist with the migration of critical corporate applications and processes to private public and hybrid cloud com environments a cloud engineer's job is to access a company's id infrastructure and migrate certain processes and operations of the cloud public private and hybrid cloud computing systems are among these functions and operations now that we know who a cloud engineer is let's have a look at the skills required to become one cloud engineers must have a diverse set of technical talents that they will use in a variety of positions several key skill sets that cloud engineers need to be successful include the ability to program and code cloud-based applications and processes the ability to troubleshoot technical challenges through problem solving the ability to plan and design software and cloud applications using critical and creative thinking should also include data analysis skills and should be able to strategically plan and to fulfill projects using technical resources now let's have a look at the roles and responsibilities of cloud engineer there are three types of cloud engineers in particular in a small business one individual may be in charge of all three roles cloud architect cloud software engineer and cloud security engineer cloud architect and organizations cloud computing strategy include application design management and monitoring is overseen by a cloud architect cloud architects connect various aspects of cloud computing including data and networks with tools and services cloud software engineer a cloud software engineer creates software for cloud computing systems and is in charge of managing their development and maintenance cloud security engineer control based technologies and procedures that enable regulatory compliance regulations are included in security when dealing with the cloud the major purpose is to protect information data data applications and the infrastructure now let's focus on the roadmap to become a cloud engineer cloud engineers generally begin their careers with a four-year degree in computer technology and the steps below defined fundamental part to becoming a cloud engineer obtain a bachelor's degree in software and systems infrastructure computing or other technical areas your bachelor's degree program should emphasize software and system infrastructure computing and other technical courses a bachelor's degree in a computer or technology related field will also qualify you for advanced degrees if you choose to pursue them learn programming languages python c plus plus java and ruby for example are great options to start with aside from programming languages you'll want to brush up on your knowledge and skills with some of the most popular cloud services including aws hadoop and azure it's important that you gain experience in cloud computing why internships and open source projects this will help you build a portfolio of work that you can use to demonstrate your competence in future job interviews and it will also teach you vital skills that you'll need to need on job consider a master's degree to enhance your cloud computing abilities while also advancing your career because of their higher level of education cloud engineers with a master's degree in a tech subject may have additional career prospects a master's degree in software engineering or system engineering for example will help you develop more of these technical abilities and expertise advancing your career get cloud computing certification to demonstrate your understanding and skill level in cloud engineering you can also check out our official website for cloud computing certification it will help you enhance your knowledge in this particular field now that we know everything about cloud computing and how to become a cloud engineer let's have a quick look at the average salary of a cloud engineer in india and in the usa in india the average salary of a cloud engineer is rupees 12 lakh 41 000 per annum and in the usa the average salary of a cloud engineer is 128 837 dollars per annum now cloud engineers may earn a substantial income with a national average salary but it can vary though depending on your location and how many years you have been in the field for example a cloud engineer with a master's degree and several years of experience may make more than a new graduate with a four-year degree who's just starting out in their career now let's dive down and explore the companies hiring cloud engineers cloud computing is gradually establishing itself as the industry standard for data storage and administration by 2022 according to ivc over a million cloud computing jobs would be generated in india amazon ibm vipro infosys oracle cisco these are the few top companies who are some of the top recruiters in this field every journey begins with a solitary foundational level certification and for aws that starting point is the aws certified cloud practitioner hello everyone i'm shanley and i welcome you all to this new video of simply learn on how to become an aws certified cloud practitioner the aws certified cloud practitioner certification gives you a high level overview of aws it doesn't go into a great detail about any one service instead focuses on the overall structure of aws and a basic understanding of what the various servers accomplish so in this video i will take you through the path of becoming a successful aws certified cloud practitioner starting with what is an aws cloud practitioner followed by the eligibility criteria for the exam then we will discuss the exam objectives then we will look into the exam content followed by how to prepare for the exam then we will have a look at some of the sample questions for your reference and then we'll discuss some tips you should remember while taking the exam so without any further delay let's get started with the video so what is an aws cloud practitioner the aws certified cloud practitioner certification is a foundational level test designed for people who can successfully show an overall understanding of the aws cloud according to amazon the skills necessary for specific employment types such as developers administrators and solution architects are not assessed by this certification aws recommends that you have at least six months of aws cloud experience now let's look into the eligibility criteria for the exam to proceed further anyone with a basic understanding of the aws platform should take the exam we recommend that you have the following factors before taking this exam first six months of aws cloud experience second understanding id services and how they are used in the aws cloud platform is a must and third is basic understanding of aws services and use cases invoicing and pricing mechanisms security concepts and how the cloud affects your organization moving on let's have an overview of the exam the exam consists of 65 questions and has a 90 minute time restriction to pass the exam you must get at least 700 out of 1000 points that is 70 percent the exam has multiple choice one write response from four possibilities and multiple response questions two correct responses from five options the cost of applying for the exam is hundred dollars and the certification needs to be renewed after every two years now that we know of the eligibility criteria for the exam let's move ahead and have a look at its objectives it will enable you to articulate the value preposition of aws cloud next you will be able to define the aws cloud and aws global infrastructure with it also it will help you understand and explain the fundamental aws cloud architectural ideas it will enable you to describe the aws platform's essential security and compliance features as well as the shared security model you'll be able to define invoicing account management and price models with it and it will also enable you to describe the fundamental code properties of aws cloud deployment and operation now let's proceed further and have a look at the content for the exam for better and organized preparation for the exam the required knowledge is divided into four domains for the exam these are various objectives within each test domain that broadly reflect the knowledge and expertise required to pass the exam cloud concept this domain makes up 28 percent of the exam security this domain makes up 24 of the exam technology makes 36 percent of the exam and billing and pricing makes up twelve percent of the exam the following three goals are included in cloud concepts they find the aws cloud and the value proposition it provides recognize characteristics of the aws cloud's economics and make a list of the various cloud architecture design principles you should be able to explain the advantages of public cloud services and define the services that are offered on aws like iaas eas and saas you must comprehend the financial benefits of cloud computing and the distinction between c a p e x and o p e x this relates to item one you should be familiar with cloud architectural design principles such as loose coupling scaling bootstrapping and automation to mention a few now comes security you should be familiar with the aws shared responsibility model which specifies who is in charge of which component of the technology stack from the data center to servers firewall rules and data encryption aws offers tools and services for deploying security evaluating your security posture and creating alert and compliance reports you must be have a thorough understanding of these services and tools in order to describe their use and benefits now comes technology you must know what the basic aws services are and how they are used you usually don't need a thorough understanding of a services specifics but you do need to comprehend its goal benefits and use cases ec2 ecs lambda light sale ebs efs s3 rds dynamodb redshift elastic load balancing auto scaling cloudfront root s3 cloudwatch cloudtrail and sms are all core services you should be familiar with the aws cloud's worldwide infrastructure regions availability zones and edge locations are all part of this then comes billing and pricing the majority of aws services are available on a pay-per-use basis however there are also alternators to save money by signing one or three year contract with a variety of payment options you must comprehend these models and the services to which they apply make sure you know what aws will charge you and what will be free inbound data transport for example is free whereas outbound data transfer is usually expensive vpc cloud formation and iam are all free services but the resources you generate with them may not be you must be aware of the potential for cause to arise now let's have a look at how to prepare for the exam step 1 start with aws training classes aws cloud practitioner essentials aws cloud practitioner technical essentials and cloud business essentials are a few courses that will help you build your knowledge in aws step 2 go through exam guides for aws certified loud practitioner certification because they give you an idea of important concepts to focus on for the exam as well as give you an overview of all the exam objectives and the pattern of questions step 3 get familiar with subject areas before the exam analyze the important subjects for the exam and keep polishing your knowledge in that area by constant revisions step 4 self sadhu to build your strength self study is ample to clear this exam just go through the aws training go through the study material available online and keep revising step 5 examine sample questions and take a free practice test practice sample papers from various website present online to see where you stand and how much you know and how much time you take to keep a track of your performance and improve yourself and 6. schedule the exam and get certified once you feel you are ready prepared and confident enroll for the exam choose a center near you for the aws child practitioner exam website and just register yourself so let me give you a glimpse of some sample questions based on each domain first question which feature of aws allows you to deploy a new application for which the requirements may change over time option 1 elasticity option 2 fault tolerance option 3 disposable resources and option 4 high availability the answer is option 1 you can deploy your app without worrying about whether it will require more or less resources in the future thanks to elasticity infrastructure can scale up and down on demand thanks to elasticity question 2 which ews service is used to enable multi-factor authentication option 1 amazon stf option 2 aws iam option 3 amazon ec2 and option 4 aws kms well the answer is option 2 iam is used to manage multi-factor authentication and to securely regulate individual and group access to aws resources question three a company would like to maximize their potential volume and array discounts across multiple accounts and also apply service control policies on member accounts what can they use to gain these benefits option 1 aws budgets option 2 aws cost explorer option 3 aws iam option 4 aws organizations well the answer is option 4 aws organizations allow you to create groups of aws accounts and then administer policies across all of them from a single location consolidated billing is available in both features set of aws organizations allowing you to set up a single payment method in the organization master account while still receiving invoices for individual activity in each member account question 4 what advantages do you get from using the aws cloud you have to choose any two option one trade capital expense for variable expense option two stop guessing about cap capacity option three increase capital expenditure option four gain greater control of the infrastructure layers and option five comply with all local security compliance programs well the answer is option one and two you may pay on a variable opex basis for the resources you use and draw on demand using public cloud services like aws so you never have to guess how much resources you need to deploy question five under the aws shared responsibility model what is the customer responsible for choose to option one physical security of the data center option two replacement and disposable of this driver option three configuration of security groups option four patch management of infrastructure and option 5 encryption of customer data the answer is option 3 and 5. customers are responsible for creating security groups network acls updating their operating systems and encrypting their data while aws is responsible for the physical security of the dc the replacement of obsolete disk drivers and infrastructure patch management question six what are two ways an aws customer can reduce their monthly spend choose two option one turn off your resources that are not being used option two use more power efficient instance type option three reserve capacity where suitable option four be efficient with the usage of security groups and option five reduce the amount of data in address charges the answer is option 1 and 3 turning of resources that aren't in use can help you save money reserve instances can also help you save money on a monthly basis without committing to a one or three year contract which is ideal for predictable workloads question 7 what are the advantages of availability zones choose to option one they allow regional disaster recovery option two they provide fault isolation option three they enable the caching of data for faster delivery to end users option 4 they are connected by low latency network connection and option 5 they enable you to connect your on-premises networks to aws to form a hybrid cloud the answer is option 2 and 4. each aws region has numerous availability zones which are unique locations each az is designed to be isolated from other failures a data center is an az and in some situations an ez is made up of numerous data centers with an area az provides low cost low latency network connectivity to other zone within that region this allows you to synchronizationly duplicate your data between data centers allowing the failover to be automated and visible to your users question 8 which aws support plans provide support via email chat and phone choose any two option 1 basic option to business option 3 developer option 4 global and option 5 enterprise the answer is option 2 and 5. only the business and enterprise programs offer email chat and phone assistance now that we have discussed all the important aspects required for aws cloud practitioner exam let me give you some tips you should remember while taking the exam first start by answering questions you are familiar with to prevent spending time on tough ones it will give you confidence as well as save you from wasting your time on trying tough questions second keep track of your time and learn how to manage it in order to complete all the questions distribute your time according to the time available to the number of questions so that you can work on each question precisely third analyze the question first then respond as there may be traps in the questions focus on keywords and try to read them two to three times for better understanding sometimes it is said that answer lies in the question itself fourth because there may be numerous answers to a question read and choose your answers carefully if you feel there is a chance that multiple answers are correct for one question then do go for it and analyze it wisely before coming to that conclusion well this was all about it today i'm going to tell you how you can become an aws solutions architect so who is an aw solutions architect now the main role of an aws solutions architect is to help you deploy your applications on the aws platform now this is nothing but a cloud computing platform so it's not to say that you can't deploy your applications on the cloud computing platform yourself it's just that when it comes to organizations the applications that you need to deploy become a whole lot more complex that's where an aw solutions architect can help ever since cloud computing became a thing companies around the world have started migrating their physical infrastructure onto the cloud that's what an aws solutions architect does they help you migrate your physical infrastructure onto the aws cloud companies around the world work on a budget an aws solutions architect will help design a cloud infrastructure based on the organization's budget before that can be done however an aw solutions architect has to create a design with an intricate and detailed blueprint of the cloud infrastructure that they plan to set up now aw solutions architects also have to focus mainly on non-functional requirements like usability reliability scalability and performance of the cloud infrastructure they're also responsible when it comes to minimizing risks that an organization can face when it comes to cloud computing platforms now they could face risks like security leaks calculation mistakes and application down times an aw solutions architect has to ensure that these don't happen before we can talk about how you can become an aws solutions architect i have some exciting news for you guys we've launched our own youtube community you'll get to see a lot of quizzes polls offers and much more to make your learning experience a whole lot more fun you can find all of this on your subscription feed or you can also click on the top right corner right now to get started and let's get back to how can you become an aws solutions architect now to become an ew certified solutions architect you need to clear the aw certified solutions architect associate level examination now here's some details about it the exam score ranges from 100 to 1000 marks and the minimum passing score is 720. however there's a catch the passing marks are actually set using statistical analysis so they can be changed based on how difficult the examination actually is the exam fee is 150 and you can also take a practice examination which costs 20 now regardless of the examination you take be it solutions architect sysops administrator or developer any associate level examination costs 150 dollars for the professional level examination it's 300 now if you want to learn more about the aws certifications i suggest you click on the top right corner and watch our video aws certifications in 10 minutes to learn more now the exam duration is of 130 minutes and you have two types of questions multiple choice and multiple answer now the multiple choice questions have four options on which one of them is right and you have multiple answer where you have five options out of which two of them are correct you can take the examination in english japanese korean and simplified chinese now let's talk about how you can schedule an examination first let's go to google and search for aws certifications click on the first link so on this page we can go to the bottom and find the different certifications aws provides click on architect and select the aws certified solutions architect associate certification click on register now and here you need to click on the aw certification account you can sign in with your amazon account or create a new one i already have an amazon account that i'll be using for signing in here in case you don't you can click on the create your amazon account button here and create an account for yourself now after that is done you can schedule a new examination and you can scroll down to the bottom here aw certified solutions architect associate level certification click on schedule exam press continue and here you need to select your language which i'm assuming is english i'm from india so i don't need to change anything here but you can change your country your time zone and other details based on your requirement and select your preferred month i want to do it in october now select search for exam center select the one that's closest to you i'm going to select this one and here you can select the day that you want to take your test and the time available i want to do it on 15 select the time that i want and press continue now you can go through all the details here change them if you want to otherwise you can press continue after this is done close and here we have the final step which is the face you can enter your details here and pay now to finish the process now let's look at an outline of the exam content now what you're going to see in the exam are five major domains and here you have each of the domains with their respective weightages first you have at 34 designing resilient architectures at 24 percent you have both defining performant architectures and specifying secure applications and architectures at 10 percent you have designing cost optimized architectures and at six percent you have defining operationally excellent architectures now let's look at each of these domains in detail now the first domain or design resilient architectures can be divided into four parts firstly you need to know how you can choose reliable or resilient storage using services like aws s3 aws glacier and aws ebs then you have how you can design decoupling mechanisms using aws services now this is possible with the help of aws sns now these aren't the only services that enable this these are just some of the services then how you can design a multi-tier architecture solution now this is important because you need to know how you can create a solution that involves several other services then you need to know how you can design highly available and fault tolerant architectures now for the second domain defining performance architectures first you have how to choose performance storages and databases services that are used are aws rds aws redshift and aws dynamodb the second step is how you can apply caching to improve performance a service that can be used for this is aws elastic cache third how you can design solutions with elasticity and scalability you have aws lambda aws cloudwatch and aws data pipeline now for the third domain which is specifying secure applications and architectures you need to know how you can secure applications using services like aws inspector aws cloudtrail and aws iam you need to know how to secure data using cloud hsm and aws macy and how you can define the networking infrastructure using front vpc and elastic load balancer for the fourth domain you have designing cost optimized architectures firstly you need to know how you can design cost optimized compute solutions you can use aws ec2 elastic bean stock lambda and aw slide sale then you need to know how you can design cost optimized storage solutions using aws s3 glacier ebs and elastic file system and the final domain to define operationally excellent architectures you need to set up design features and solutions that enable operational excellence now some of the features are that you perform operations as code you annotate documentation you make frequent and small reversible changes and anticipate and tackle failures now let's have a look at the job opportunities when it comes to aws solutions architects now if you have a look at this graph you can see that aws has always provided a lot more job postings as compared to the other two giants in the cloud computing domain which are microsoft azure and google cloud platform now if you do a little bit of research you'll find out that there's a significant lacking of experienced personnel when it comes to aws so i would suggest this is the best time for you to get certified in aws the amazon web services cloud management system now has over a million clients and generates 10 billion dollar in revenue annually kellogg's samsung unilever nokia netflix adobe are the majority of international corporation trust the platform so hello everyone i'm shamley and i'm going to take you through this new video of simply learn on top 10 reasons to choose aws through the medium of this video i'm going to give you top 10 reasons to choose adapt learn and explore aws but before moving ahead let me give you an overview of the concept of aws for your better understanding amazon web services aws is a cloud computing platform that manages and maintains hardware and infrastructure reducing the expense and complexity of purchasing and running resources on site for businesses and individuals these resources are available for free or for a fee per usage it is a comprehensive and simple to use computing platform offered by amazon infrastructure as a service platform as a service and packet software as a service are all used to build the platform so let's move ahead and explore the top 10 reasons to choose aws starting with the first one which is flexible pricing options the most popular motivation for learning aws is because of its cost-effective pricing alternatives aws provides a year of free access to the free tire which is idle for novices who wish to learn about aws technology for beginners we believe this is plenty aws is highly customizable with pay as you go options to match any company need flexible pricing is a great approach to scale up or down the architecture as needed the aws tariffs are comparable to those for water and electricity second reason to proceed with is the global architecture one of the most appealing features of aws is that it is offered in 24 different zones across 16 different countries you can connect to the server from any country you may use all of the advantages by hiring an aws developer from anywhere there are no restrictions on who can use the servers based on the location third is automated multi-region backups aws offers a variety of backup options including amis and ebs snapshots aws's decentralized global reach makes storing crucial data in several geographical locations simple and cost effective as a result if your main production environment is brought down by a natural or man-made calamity your backup data will be unaffected third-party solutions can also let businesses automate backups between aws sites without the need for internal scripting fourth reason that we will come across is flexibility and scalability one of the most valuable aws features is its flexibility it lets you pick your operating system programming language web application platform database and other services aws provides a virtual environment in which your application may load all of the services it requires it also makes data migration easier when demand increases or drops auto scaling and elastic load balancing techniques are automatically scaled up or down via aws aws approaches are useful for dealing with unpredictably high or unpredictable loads organizations gain from lower cost and more user satisfaction as a result of this moving on fifth is streamlined recovery even a small bit of downtime or data loss might be disastrous for some services for others the cost of downtime data loss is not greater than the expense of maintaining a multi-site hot standby recovery strategy regardless of your organization's resilience to downtime or data loss aws adaptable platform can provide you with the tools you need to implement a disaster recovery strategy in the event of a disaster aws disaster recovery restores your data fast across several locations sixth one is pass offering the most prevalent model is platform as a service which has a market share of roughly 32 and is likely to rise in the next few years a platform as a service solution provides a platform for developers to construct unique configurable software and is a popular choice for enterprises looking to create unique apps without spending a fortune or shouldering all of the responsibilities seventh reason i will take you through is customization customization for aws control car is a solution that combines aws control target other highly available trusted aws services to assist customers set up a secure multi-account aws environment faster and more efficiently using aws best practices customers must first have an aws control tower landing zone set up in the account before installing the solution eighth one is scheduling aws clients can start and stop services at any moment thanks to its service scheduling feature you can keep track of your assets pay for the service you use and run them at a certain time it enables you to create sample planners to test consumers viability and schedule services as required consider elastic computing cloud ec2 and relational database service as examples of scheduling services you are not required to conduct these services on a daily basis set the time table for the service based on the day of the week you utilize it ninth is security aws provides the same level of world-class security to small one startups as well as enterprise size behemoths the commercial data center meets the highest conceivable standards saving you the trouble of needing to defend your own data centers aws also maintains a large number of compliance processes in its infrastructure as well as a large security help network that may provide real-time information about suspicious stars and potential vulnerabilities and the last one the 10th reason is third party apis either blues api implies that you can manage your cloud-based facilities in a variety of languages in line with the platform's entire versatility it also means that third-party solutions such as yours can take advantage of all of aws time and cost savings feature you can automate ordinary but necessary aws processes without the use of coding from automated backups to ec2 and rds instances well this brings us to the end of this video so how can simply learn help you with your aws certifications now we are on the sampling on home page and here we have the awa solutions architect course on this course which is the aws solutions architect certification training course we have 36 hours of instructor-led training 20 hours of self-paced learning 16 live demos three simulation exams three hands-on practice projects and so much more here our main emphasis is on making sure that you have enough hands-on experience with the services that you crack the examination at your first attempt so here we also go through some of the more important services that you need to learn like iam vpc ec2 s3 and so much more we'll also talk about databases application services security practices disaster recovery and so on we even have practice examinations to help you prepare for the real one choose from over 300 in-demand skills and get access to 1 000 plus hours of video content for free visit scale up by simply learn click on the link in the description to know more i'm here to walk you through some of the aws interview questions which we find are important and our hope is that you would use this material in your interview preparation and be able to crack that cloud interview and step into your dream cloud job by the way i'm and cloud technical architect trainer and an interview panelist for cloud network and devops so as you progress in watching you're going to see that these questions are practical scenario-based questions that tests the depth of the knowledge of a person in a particular aws product or in a particular aws architecture so why wait let's move on all right so in an interview you would find yourself with a question that might ask you define and explain the three basic types of cloud services and the aws products that are built based on them see here it's a very straightforward question just explain three basic types of cloud service and when we talk about basic type of cloud service it's compute obviously that's a very basic service storage obviously because you need to store your data somewhere and networking that actually connects a couple of other services to your application these basic will not include monitoring these basic will not include analytics because they are considered as optional they are considered as advanced services you could choose a non-cloud service or a product for monitoring of and for analytics so they're not considered as basic so when we talk about basics they are compute storage and networking and the second part of the questions is explain some of the aws products that are built based on them of course compute ec2 is a major one that's that's the major share of the compute resource and then we have platform as a service which is elastic bean stock and then function as a service which is lambda auto scaling and light cell are also part of compute services so the compute domain it really helps us to run any application and the compute service helps us in managing the scaling and deployment of an application again lambda is a compute server so the compute service also helps in running event initiated stateless applications the next one was storage a lot of emphasis is on storage these days because if there's one thing that grows in a network on a daily basis that storage every new day we have new data to store process manage so storage is again a basic and an important cloud service and the products that are built based on the storage services are s3 object storage glacier for archiving ebs elastic block storage as a drive attachment for the ec2 instances and the efs file share for the ec2 instances so the storage domain helps in the following aspects it holds all the information that the application uses so it's the application data and we can also archive old data using storage which would be glacier and any object files and any requirement for block storage can be met through elastic block store and s3 which is again an object storage talking about uh networks it's just not important to answer the question with the name of the services and the name of the product it'll also be good if you could go in depth and explain how they can be used right so that actually proves you to be a person knowledgeable enough in that particular service or product so talking about networking domain vpc networking can't imagine networking without vpc in in the cloud environment especially in aws cloud environment and then we have route 53 for domain resolution or for dns and then we have cloudfront which is an edge caching service that helps customers get our customers to read their application with low latency so networking domain helps with some of the following use cases it controls and manages the connectivity of the aws services within our account and we can also pick an ip address range if you're a network engineer or if you are somebody who works in networks or are planning to work the network you will soon realize the importance of choosing your own ip address for easy remembering so having an option to have your own ip address in the cloud on the range of ip address in the cloud it really helps really really helps in cloud networking the other question that get asked would be the difference between the availability zone and the region actually the question generally gets asked so to test how well you can actually differentiate and also correlate the availability zone and the region relationship right so a region is a separate geographic area like the us west one i mean which represents north california or the ap south which represents mumbai so regions are a separate geographic area on the contrary availability zone resides inside the region you shouldn't stop there you should go further and explain about availability zones and availability zones are isolated from each other and some of the services will replicate themselves within the availability zone so availability zone does replication within them but regions they don't generally do replication between them the other question you could be asked is what is auto scaling what do we achieve by auto scaling so in short auto scaling it helps us to automatically provision and launch new instances whenever there is an demand it not only helps us meeting the increasing demand it also helps in reducing the resource usage when there is low demand so auto scaling also allows us to decrease the resources or resource capacity as per the need of that particular r now this helps business in not worrying about putting more effort in managing or continuously monitoring the server to see if they have the needed resource or not because auto scaling is going to handle it for us so business does not need to worry about it and auto scaling is one big reason why people would want to go and pick a cloud service especially an aws service the ability to increase and shrink based on the need of that art that's how powerful is auto scaling the other question you could get asked is what's your targeting in cloud front now we know that cloudfront is caching and it caches content globally in the amazon caching service global wide the whole point is to provide users worldwide access to the data from a very nearest server possible that's the whole point in using or going for cloud front then what do you mean by geo targeting jio targeting is showing customer and specific content based on language we can customize the content based on what's popular in that place we can actually customize the content the url is the same but we could actually change the content a little bit not the whole content otherwise it would be dynamic but we can change the content a little bit a specific a file or a picture or a particular link in a website and show customized content to users who will be in different parts of the globe so how does it happen cloudfront will detect the country where the viewers are located and it will forward the country code to the origin server and once the origin server gets the specialized or a specific country code it will change the content and it'll send to the caching server and it get cached there forever and the user gets to view a content which is personalized for them for the country they are in the other question you could get asked is the steps involved in using cloud formation or creating a cloud formation or backing up an environment within cloud formation template we all know that if there is a template we can simply run it and it provisions the environment but there is a lot more going into it so the first step in moving towards infrastructure as a code is to create the cloud formation template which as of now supports json and yaml file format so first create the cloud formation template and then save the code in an x3 bucket s3 bucket serves as the repository for our code and then from the cloud formation call the file in the s3 bucket and create a stack and now cloud formation uses the file reads the file understands services that are being called understands the order understands how they are connected with each other cloud formation is actually an intelligent service it understands the relation based on the code it would understand the relationship between the different services and it would set an order for itself and then would provision the services one after the other let's say a service has a dependency and the dependent service the other service which this service let's say service a and b service b is dependent on service a let's say cloud formation is an intelligent service it would provision the resource a first and then would provision resource b what happens if we inverse the order if we inverse the order resource b first gets provision and because it does not have dependency chances that the cloud formation's default behavior is that if something is not provisioned properly something is not healthy it could roll back chances that the environment provisioning will roll back so to avoid that cloud formation first provisions all the services that has or that's dependent on that's depended by another service so it provisions those service first and then provisions the services that has dependencies and if you are being hired for a devops or you know if the interviewer wanted to test your skill on a systems side this definitely would be a question in his list how do you upgrade or downgrade a system with near zero downtime now everybody's moving towards zero downtime or near zero downtime all of them want their application to be highly available so the question would be how do you actually upgrade or downgrade a system with near zero downtime now we all know that i can upgrade an ec2 instance to a better ec2 instance by changing the instance type stopping and starting but stopping and starting is going to cause a downtime right so that's you should be answering or you shouldn't be thinking in those terms because that's the wrong answer specifically the interviewer wants to know how do you upgrade a system with zero downtime so upgrading system with zero downtime it includes launching another system parallelly with the bigger ec2 instance type over the bigger capacity and install all that's needed if you're going to use an ami of the old machine well and good you don't have to go through installing all the updates and installing all the application from the ami once you've launched it in a bigger instance locally test the application to see if it is working don't put it on production yet test the application to see if it is working and if the application works we can actually swap if your server is behind and behind route 53 let's say all that you could do is go to rot 53 update the information with the new ip address new ip address of the new server and that's going to send traffic to the new server now so the cutover is handled or if you're using static ip you can actually remove the static ip from the old machine and assign it to the new machine that's one way of doing it or if you are using elastic nic card you can actually remove the new card from the old machine and attach the new card to the new machine so that way we would get near zero downtime if you're hired for an architect level you should be worrying about cost as well along with the technology and this question would test how well you manage cost so what are the tools and techniques we can use in aws to identify and correct identify and know that we are paying the correct amount for the resources that we are using or how do you get the visibility of your aws resources running one way is to check the billing there is a place where you can check the top services that were utilized it could be free and it could be paid service as well top services that can be utilized it's actually in the dashboard of the cost management console so that table here shows the top five most used services so looking at it you can get it all right so i'm using a lot of storage i'm using a lot of ec2 why is storage high you can go and try to justify that and you will find if you are storing things that should be storing and then clean it up why is compute capacity so high why is data transfer so high so if you start thinking in those levels you'll be able to dig in and clean up unnecessary and be able to save your bill and there are cost explorer services available which will help you to view your usage pattern or view your spending for the past 13 months or so and it will also forecast for the next three months now how much will you be using if your pattern is like this that will actually help and will give you a visibility on how much you have spent how much you will be spending if the trend continues budgets are another excellent a way to control cost you can actually set up budget alright this is how much i am willing to spend for this application for this team or for this month for this particular resource so you can actually put a budget mark and anytime it exceeds any time it's nearing you would get an alarm saying that well we're about to reach the allocated budget amount stuff like that that way you can go back and know and you know that how much the bill is going to be for that month or you can take steps to control bill amount for that particular month so aws budget is another very good tool that you could use cost allocation tags helps in identifying which team or which resource has spent more in that particular month instead of looking at the bill as one list with no specifications into it and looking at it as an expenditure list you can actually break it down and tag the expenditure to the teams with cost allocation tax the dev team has spent so much the production team has spent so much the training team has spent more than the dev and the production team why is that now you'll be able to you know think in those levels only if you have cost allocation tax now cost allocation tax are nothing but the tags that you would put when you create a resource so for production services you would put as a production tag you would create a production tag and you would associate that resources to it and at a later point when you actually pull up your bill that's going to show a detailed a list of this is the owner this is the group and this is how much they have used in the last month and you can move forward with your investigation and encourage or stop users using more services with the cost allocation tax the other famous question is are there any other tools or is there any other way of accessing aws resource other than the console console is gui right so in other words other than gui how would you use the aws resource and how familiar are you with those tools and technologies the other tools that are available that we can leverage and access the aws resource are of course hoodie you can configure pudi to access the aws resources like log into an ec2 instance and ec2 instance does not always have to be logged in through the console you could use putty to log into an ec2 instance like the jump box like the proxy machine and like the gateway machine and from there you can actually access the rest of the resources so this is an alternative to the console and of course we have the aws cli in any of the linux machines or windows machines we can install so that's point two three and four we can install aws cli for linux windows also for mac so we can install them and from there from your local machine we can access run aws commands and access provision monitor the aws resources the other ones are we can access the aws resource programmatically using aws sdk and eclipse so these are a bunch of options we have to use the aws resource other than the console if you're interviewed in a company or by a company that focuses more on security and want to use aws native services for their security then you would come across this question what services can be used to create a centralized logging solution the basic services we could use our cloud watch logs store them in s3 and then use elasticsearch to visualize them and use kinesis to move the data from s3 to elasticsearch right so log management it actually helps organizations to track the relationship between operational and security changes and the events that got triggered based on those logs instead of logging into an instance or instead of logging into the environment and checking the resources physically i can come to a fair conclusion by just looking at the logs every time there's a change the system would scream and it gets tracked in the cloud watch and then cloud watch pushes it to s3 kinesis pushes the data from s3 to elasticsearch and i can do a time-based filter and i would get an a fair understanding of what was going on in the environment for the past one or whatever the time window that i wanted to look at so it helps in getting a good understanding of the infrastructure as a whole all the logs are getting saved in one place so all the infrastructure logs are getting saved in one place so it's easy for me to look at it in an infrastructure perspective so we know the services that can be used and here are some of the services and how they actually connect to each other it could be logs that belongs to a one account it could be logs that belongs to multiple accounts it doesn't matter you know those three services are gonna work fairly good and they're gonna inject or they're gonna like suck logs from the other accounts put it in one place and help us to monitor so as you see you have cloud watch here that actually tracks the metrics you can also use cloud trail if you want to log api calls as well push them in an s3 bucket so there are different types of blog flow logs are getting captured in an instance application logs are getting captured from the same vpc from a different vpc from the same account from a different account and all of them are analyzed using elasticsearch using the kibana client so step one is to deploy the ecs cluster step two is to restrict access to the ecs cluster because it's valid data you don't want anybody to put their hands and access their data so resting access to the ecs dashboard and we could use lambda also to push the uh data from cloudwatch to the elasticsearch domain and then kibana is actually the graphical tool that helps us to visualize the logs instead of looking at log as just statements or in a bunch of characters a bunch of files kibana helps us to analyze the logs in a graphical or a chart or a bar diagram format again in an interview the interviewer is more concerned about testing your knowledge on aw security products especially on the logging monitoring even management or incident management then you could have a question like this what are the native aws security logging capabilities now most of the services have their own logging in them like have their own logging like s3 s3 has its own login and cloudfront has its own logging ds has its own login vpc has its own logging in additional there are account level logins like a cloudtrail and aws config services so there are variety of logging options available in the aws like cloud soil config cloudfront redshift logging rds logging vpc flow logs s3 object logging s3 access logging stuff like that so we're going to look at two servers in specific cloud trail now this cloud trail the very first product in that picture we just thought the cloud trail provides an very high level history of the api calls for all the account and with that we can actually perform a very good security analysis a security analysis of our account and these logs are actually delivered to you can configure it they can be delivered to s3 for long time archivals and based on a particular event it can also send an email notification to us saying hey just got this error thought i'll let you know stuff like that the other one is config service a config service helps us to understand the configuration changes that happened in our environment and we can also set up notifications based on the configuration changes so it records the cumulative changes that are made in a short period of time so if you want to go through the lifetime of a particular resource what are the things that happened what are the things it went through they can be looked at using aws config all right the other question you could get asked is if you know your role includes taking care of cloud security as well then the other question you could get asked is the native services that amazon provides to mitigate ddos which is denial of service now not all companies would go with amazon native services but there are some companies which want to stick with amazon native services just to save them from the headache of managing the other softwares or bringing in another tool a third-party tool into managing ddos they simply want to stick with amazon proprietary amazon native services and a lot of companies are using amazon service to prevent ddos denial of service now denial of service is if you already know what denial of service is well and good if you do not know then let's know it now denial of service is a user trying to or maliciously making attempt to access a website or an application the user would actually create multiple sessions and he would occupy all the sessions and he would not let legitimate users access the servers so he's in turn denying the service for the user a quick picture review of what denial of services now look at it these users instead of making one connection they are making multiple connections and there are cheap software programs available that would actually trigger connections from different computers in the internet with different mac addresses so everything kind of looks legitimate for the server and it would accept those connections and it would keep the sessions open the actual users won't be able to use them so that's denying the service for the actual users denial of service all right and distributed denial of service is generating attacks from multiple places you know from a distributed environment so that's distributed denial of service so the tools the native tools that helps us to prevent the denial of service attacks in aws is cloud shield and web access firewall aws now they are the major ones they are designed to mitigate a denial of service if your website is often bothered by denial of service then we should be using aws shield or aws waff and there are a couple of other tools that also does when i say that also does denial of service is not their primary job but you could use them for denial of service route 53's purpose is to provide dns cloudfront is to provide caching elastic load balancer elbs work is to provide load balancing vpc is to create and secure a virtual private environment but they also support mitigating denial of service but not to the extent you would get in aws shield and aws web so aws shield and waff are the primary ones but the rest can also be used to mitigate distributed denial of service the other tricky question is this actually will test your familiarity with the region and the services available in the region so when you're trying to provision a service in a particular region you're not seeing the service in that region how do we go about fixing it or how do we go about using the service in the cloud it's a tricky question and if you have not gone through such situation you can totally blow it away you really need to have a good understanding on regions the services available in those regions and what if a particular service is not available how to go about doing it the answer is not all services are available in all regions anytime amazon announces a new service they don't immediately publish them on all regions they start small and as in when the traffic increases as and when it becomes more likeable to the customers they actually move the service to different regions so as you see in this picture within america not virginia has more services compared to ohio or compared to north california so within not america itself north virginia is the preferred one so similarly there are preferred regions within europe middle east and africa and preferred regions within asia pacific so anytime we don't see a service in a particular region chances that the service is not available in that region yet we got to check the documentation and find the nearest region that offers that service and start using the service from that region now you might think well if i'm looking for a service in asia let's say in mumbai and if it is not available why not simply switch to north virginia and start using it you could but you know that's going to add more latency to your application so that's why we need to check for application which is check for region which is very near to the place where you want to serve your customers and find nearest region instead of always going back to north virginia and deploying an application in north virginia again there's a place there's a link in aws.com that you can go and look for services available in different region and that's exactly what you're seeing here and if your service is not available in a particular region switch to the other region that provides your service the nearest other region that provides that service and start using service from there with the coming up of cloud a lot of companies have turned down their monitoring team instead they want to go with the monitorings that cloud provides you know nobody wants to or at least many people don't want to go through the hassle of at least new startups and new companies that are thinking of having a monitoring environment that they don't want to go with traditional knock monitoring instead they would like to leverage aws monitorings available because it monitors a lot of stuff not just the availability but it monitors a lot of stuff like failures errors it also triggers emails stuff like that so how do you actually set up a monitor to website how to set up a monitor to monitor the website metrics in real time in aws the simple way anytime you have a question about monitoring cloudwatch should strike your mind because cloudwatch is meant for monitoring is meant for collecting metrics is meant for providing graphical representation of what's going on in a particular network at a particular point of time so cloudwatch cloudwatch helps us to monitor applications and using cloudwatch we can monitor the state changes not only the state changes the auto scaling life cycle events anytime there are there are more services added there is a reduction in the number of servers because of less usage and very informative messages can be received through cloud watch any cloud watch can now support scheduled events if you want to schedule anything cloudwatch has an event that would schedule an action all right schedule a trigger time based not incident based you know anything happening and then you get an action happening that's incident based on the other hand you can simply schedule few things on time based so that's possible with cloud watch so this cloud watch integrates very well with a lot of other services like notifications for notifying the user or for notifying the administrator about it and it can integrate well with lambda so to trigger an action anytime you're designing an auto healing environment this cloud watch can actually monitor and send an email if we are integrating it with sns simple notification service or this cloud watch can monitor and based on what's happening it can trigger an event in lambda and that would in turn run a function till the environment comes back to normal so cloudwatch integrates well with a lot of other aw services all right so cloudwatch has uh three statuses green when everything is going good yellow when the service is degraded and red when the service is not available green is good so we don't have to do anything about it but anytime there is an ello the picture that we're looking at it's actually calling an lambda function to debug the application and to fix it and anytime there's a red alert it immediately notifies the owner of the application about well the service is down and here is the report that i have here is the metrics that i've collected about the service stuff like that if the job role requires you to manage the servers as well there are certain job roles which are on the system side there are certain job roles which is development plus system side now you're responsible for the application and the server as well so if that's the case you might be tested with some basic questions like the different types of virtualization and aws and what are the difference between them all right the three major types of virtualization are hvm which is hardware virtual machine the other one is pv para virtualization and the third one is pv on hvm para virtualization on hardware virtual module all right the difference between them or actually describing them is actually the difference between them hvm it's actually a fully virtualized hardware you know the whole hardware is virtualized and all virtual machines act separate from each other and these vms are booted by executing master boot record in the root block and when we talk about para virtualization paragraph is actually the special boot loader which boots the pv amis and when we talk about pv on hvm it's it's actually the marriage between hvm and pv and this para virtualization on hvm in other words pv on hvm it actually helps operating system take advantage in storage and the network input output available through the host another good question is name some of the services that are not region specific now you've been thought that all services are within a region and some services are within an availability zone for example ec2 is within an availability zone ebs is within an availability zone s3 is region specific dynamodb is region specific stuff like that vpc is both availability and region specific meaning you know subnets are availability zone specific and vpcs region specific stuff like that so you might have thought you might have learned in that combination but there could be some tricky questions that tests you how well you have understood the region non-region and availability non-availability services i should say there are services that are not region specific that would be iam so we can't have im for every availability zone and for every region which means you know users will have to use one username and password for one region and anytime they switch to another region they will have to use another username and password that that's more work and that's not a good design as well authentication has to be global so im is a global service and which means it's not region specific on the other hand route 53 is again a regional specific so we can't have a route 53 for every region route 53 is not a region specific service it's a global service and it's one application users access from everywhere or from every part of the world so we can't have one url or one dns name for each region if your application is a global application and then web application firewall works well with cloudfront then cloudfront is a region based service so the web application firewall it's not region specific service it's a global service and cloudfront is again a global service though you can you know cache content on a continent and country basis it's still considered a global service all right it's not bound to any region so when you activate cloud front you're activating it away from region or availability zone so when you're activating a web application firewall because it's not a region-specific service you're activating it away from availability zone and regions so a quick recap i am users groups roles and accounts they are global services they can be used globally around 53 services are offered at edge locations and they are global as well web application firewall a service that protects our web application from common web exploits they are global service as well cloudfront cloudfront is global content delivery network cdn and they are offered at edge locations which are a global service in other words non-region specific service or beyond region service all right this is another good question as well in the project that you are being interviewed if they really want to secure their environment using nat or if they are already securing their environment using nat by any of these two methods like nat gateway or nat instances you can expect this question what are the difference between and that gateway and that instances now they both saw the same thing right so they're not two different services trying to achieve two different things they both serve the same thing but still they do have differences in them right on a high level they both achieve providing netting for the service behind it but the difference comes when we talk about the availability of it nat gateway is a managed service by amazon whereas nat instance is managed by us now i'm talking about the third point maintenance here nat gateway is managed by amazon that instance is managed by us and availability of nat gateway is very high and availability of that instance is less compared to the nat gateway because it's managed by us you know it's on an ec2 instance which could actually fail and if it fails we'll have to relaunch it but if it is not gateway if something happens to that service amazon would take care of reprovisioning it and talking about bandwidth it can burst up to 75 gigabits now traffic through the net gateway can burst up to 75 gigabits but for that instance it actually depends on the server that we launched and if we are launching a t2 micro it barely gets any bandwidth so there's a difference there and the performance because it's highly available because of the bigger pipe 75 gigabits now the performance of the night gateway is very high but the performance of the nat instance is going to be average again it depends on the size of the nat instance that we pick and billing a billing for that gateway is the number of gateways that we provision and the duration for which we use the nat gateway but billing for nat instance is number of instance and the type of instance that we use of course number of instance duration and the type of instance that we use security not gateway cannot be assigned meaning it already comes with full packed security but in that instance security is a bit customizable i can go and change the security because it's a server managed by me or managed by us i can always change the security well allow this allow don't allow this stuff like that size and load of the night gateway is uniform but the size and the load of the uh that instance changes as per that gateway is a fixed product but not instance can be small instance can be a big instance so the size and the load through it varies right the other question you could get asked is what are the difference between stopping and terminating an ec2 instance now you will be able to answer only if you have worked on environments where you have your instance stopped and where you have your instance terminated if you have only used lab and are attending the interview chances are that you might you always lost when answering this question it might look like both are the same well stopping and terminating both are the same but there is a difference in it so when you stop an instance it actually performs a normal shutdown on the instance and it simply moves the instance to the stopped state but when you actually terminate the instance the instance is moved to this stop state the evs volumes that are attached to it are deleted and removed and we'll never be able to recover them again so that's a big difference between stopping and terminating an instance if you're thinking of using the instance again along with the data in it you should only be thinking of stopping the instance but you should be terminating the instance only if you want to get rid of that instance forever if you are being interviewed for an architect level position or a junior architect level position or even a cloud consultant level position or even in an engineering position this is a very common question that gets asked what are the different types of easy to instances based on their cost or based on how we pay them right they're all compute capacity for example the different types are on demand instances spot instances and reserved instances it kind of looks the same they all provide the compute capacity they all provide the same type of hardwares for us but if you're looking at cost saving or optimizing cost in our environment we got to be very careful about which one are we picking now we might think that well i'll go with on-demand instance because i pay on a per hour basis which is cheap you know i can use them anytime i want and anytime i don't want i can simply get rid of it by terminating it you're right but if the requirement is to use the service for one year the requirement is to use the service for three years then you'll be wasting a lot of money buying on-demand instances you'll be wasting a lot of money paying on an hourly basis instead we should be going for reserved instance where we can reserve the capacity for the complete one year or complete three years and save huge amount in buying reserved instances all right so under man is cheap to start with if you're only planning to use it for a short while but if you're planning to run it for a long while then we should be going for reserved instance that is what is cost efficient so spot instance is cheaper than on-demand instance and there are different use cases for spot instance as well so let's look at one after the other the on-demand instance the on-demand instance is purchased at a fixed rate per r this is very short term and irregular workloads and for testing for development on demand instance is a very good use case we should be using on demand for production spot instance spot instance allows users to purchase ec2 at a reduced price and anytime we have more instances we can always go and sell it in spot instances i'm referring to anytime we have more reserved instances we can always sell them in spot instance catalog and the way we buy spot instance is we actually put a budget this is how much i'm willing to pay all right would you be able to give service within this cost so anytime the price comes down and meets the cost that we have put in will be assigned an instance and anytime the price shoots up the instance will be taken away from us but in case of on-demand instances we have bought that instance for that particular r and it stays with us but with spot instances it varies based on the price if you meet the price you get the instance if you don't meet the price goes away to somebody else and the spot instance availability is actually based on supply and demand in the market there's no guarantee that you will get spot instance at all time all right so that's a caveat there you should be familiar with that's a caveat that you should be aware when you are proposing somebody that we can go for spot instance and save money it's not always going to be available if you want your spot instance to be available to you then we need to carefully watch the history of the price of the spot instance now how much was it last month and how much was it how much is it this month so how can i code or how much can i code stuff like that so you got to look at those history before you propose somebody that well we're going to save money using spot instance on the other hand reserved instance provide cost savings for the company we can opt for reserved instances for you know one year or three years there are actually three types of reserved instances light medium and heavy reserved instances they are based on the amount that we would be paying and cost benefit also depends with reserved instance the cost benefit also depends based on are we doing all upfront or no upfront or partial payment then split the rest as monthly payments so there are many purchase options available but overall if you're looking at using an application for the next one year and three years you should not be going for on-demand instance you should be going for reserved instance and that's what gives you the cost benefit and in an error-based interview sometimes you might be asked you know how you interact with the aws environment are you using cli are you using console and depending on your answer whether console or a cli the panelist put a score okay this person is cli specific this person is console specific or this person has used aws environment through the sdk stuff like that so this question tests whether you are a cli person or an console person and the question goes like this how do you set up ssh agent forwarding so that you do not have to copy the key every time you log in if you have used footy anytime if you want to log into an ec2 instance you will have to put the ip and the port number along with that you will have to map or we will have to map the key in the puri and this has to be done every time that's what we would have done in our lab environments right but in production environment using the same key or mapping the same key again and again every time it's actually in hassle it's considered as a blocker so you might want to cache it you might want to permanently add it in your putty session so you can immediately log in and start using it so here in the place where you would actually map the private key there's a quick button that actually fixes or that actually binds your ssh to your instance so we can enable ssh agent forwarding that will actually bind our key to the ssh and next time when we try to log in you don't have to always go through mapping the key and trying to log in all right this question what are solares and ax operating systems are they available with aws that question generally gets asked to test how familiar are you with the amis available how familiar are you with ec2 how familiar are you with the ec2 hardwares available that basically test that now the first question or the first thought that comes to your mind is well everything is available with aws i've seen windows i've seen ubuntu i've seen red hat i've seen amazon amis and if i don't see my operating system there i can always go to marketplace and try them if i don't find a marketplace i can always go to community and try them so a lot of amis available that lot of operating systems available i will be able to find solaris and ax but that's not the case solar s and ax are not available with aws that's because solaris uses a different and solaris does not support the architecture does not support public cloud currently the same goes for ax as well and they run on power cpu and not on intel and as of now amazon does not provide power machines this should not be confused with the hpc which is a high performance computing should not be confused with that now these are different hardwares different cpu itself that the cloud providers did do not provide yet another question you could get asked in organizations that would want to automate their infrastructure using amazon native services would be how do you actually recover an ec2 instance or auto recover an ec2 instance when it fails well we know that ec2 instances are considered as immutable meaning irreparable we don't spend time fixing bugs in an os stuff like that you know once an ec2 instance crashes like it goes on a always panic or there are various reasons why it would fail so we don't have to really worry about fixing it we can always relaunch that instance and that would fix it but what if it happens at two o'clock in the night what if it happens that during a weekend when nobody's in office looking or monitoring those instances so you would want to automate that not only on a weekend or during midnights but it's general practice good to automate it so you could face this question how do you actually automate an ec2 instance once it fails and the answer to that question is using cloud watch we can recover the instance so as you see there is an alarm threshold a set in cloud watch and once the threshold is met meaning if there is an error if there is a failure if the ec2 instance is not responding for a certain while we can set an alarm and once the alarm is met let's say the cpu utilization stayed high for five minutes all right it's not taking any new connections or the instance is not pinging for five minutes or in this case it's two minutes it's not pinging so it's not going to respond connection so in those cases you would want to automatically recover that ec2 instance by rebooting the instance all right now look at this the take this action section under the action so there we have a bunch of options like recover this instance meaning reboot the instance so that's how we would recover the other two options are beyond the scope of the question but still you can go ahead and apply just like i'm gonna do it so the other option is stop the instance that's very useful when you wanna stop instances that are having low utilizations nobody's using the system as of now you don't want them to be running and wasting the cloud expenditure so you can actually set an alarm that stops the ec2 instance that's having low utilization so somebody was working in an instance and they left it without or they forgot to shut down that instance and it gets i mean they will only use it again the next day morning so in between there could be like 12 hours that the system is running idle nobody's using it and you're paying for it so you can identify such instances and actually stop them when the cpu utilization is low meaning nobody is using it the other one is to terminate let's say you want to give system to somebody temporarily and you don't want them to hand the system back to you all right this is actually an idea in other words this is actually the scenario so you hand over a system to somebody and when they're done they're done we can actually terminate the system so you could instruct the other person to terminate the system when they're done and they could forget and the instance could be running forever or you can monitor the system after the specified time is over and you can terminate the system or best part you can automate the system termination so you assign a system to somebody and then turn on this cloud watch action to terminate the instance when the cpu is low for like two hours meaning they've already left or cpu is low for 30 minutes meaning they've already left stuff like that so that's possible and if you're getting hired for an system side architect or even on the sysop site you could face this question what are the common and different types of ami designs there are a lot of ami designs the question is the common ones and the difference between them so the common ones are the full back a mice and the other one is just enough os ami j e os ami and the other one is hybrid type amis so let's look at the difference between them the full backed ami just like the name says it's fully baked it's ready to use ami this is the simplest ami to deploy can be a bit expensive it can be a bit cumbersome because you'll have to do a lot of work beforehand you could use the ami so a lot of planning a lot of thought process will go into it and the ami is ready to use right you hand over the ami to somebody and it's ready to use or if you want to reuse the ami it's already ready for you to use so that's full baked ami the other one is just enough operating system ami just like the name says it has i mean as you can also see in the diagram or in the picture it covers a part of the os all bootstraps are already packed properly and the security monitoring logging and the other stuff are configured at the time of deployment or at the time you would be using it so not much thought process will go in here the only focus is on choosing the operating system and what goes the operating system specific agents or bootstraps that goes into the operating system that's all we worry about the advantage of this is it's flexible meaning you can choose to install additional softwares at the time of deploying but that's going to require an additional expertise on the person who will be using the ami so that's another overhead there but the advantage is that it's kind of flexible i can change the configurations during the time of deployment the other one is a hybrid ami now the hybrid ami actually falls in between the fully baked ami and just enough operating system options so these amis have some features of the big type and some features of the just enough os type so as you see the security monitoring logging are packed in that ami and the runtime environments are installed during the time of a deployment so this is where the strict company policies would go into the ami company policies like you gotta log this you gotta monitor this these are the ports that generally gets open in all the systems stuff like that so they strictly go into the ami and sits in an ami form and during deployment you have the flexibility of choosing the different runtime and the application that sits in an ec2 instance another very famous question you would face in an interview is how can you recover login to an ec2 instance to which you lost the key well we know that if the key is lost we can't recover it there are some organizations that integrate their ec2 instances with an 80 that's different all right so you can go and reset the password and the ad and you will be able to log into the new password but here the specific tricky question is you are using a key to log in and how do you recover if you have lost the key generally companies would have made a backup of the key so we can pick from the backup but here the specific question is we have lost the key literally no backups on the key at all so how can we log in and we know that we can't log into the instance without the key present with us so the steps to recover is that make the instance use another key and use that key to log in once the key is lost it's lost forever we won't be able to recover it you can't raise the ticket with amazon not possible they're not going to help it's beyond the scope so make the instance use another key it's only the key that's the problem you still have valid data in it you got to recover the data it's just the key that's having the problem so we can actually focus on the key part alone and change the key and that will allow us to log in so how do we do it step by step procedure so first verify the easy to config services running in that instance if you want you can actually beforehand install the easy to config in that service or you can actually make the easy to config run through the console just a couple of button clicks and that will make the easy to configure run in that ec2 instance and then detach the root volume for that instance of course it's going to require a stop and start to detach the root volume from the instance attach the root volume to another instance as a temporary volume or it could be a temporary instance that you've launched only to fix this issue and then login to that instance and to that particular volume and modify the configuration file configuration file modify it to use the new key and then move the root volume back to its original position and restart the instance and now the instance is going to have the new key and you also have the new key with which you can log in so that's how we go ahead and fix it now let's move on to some product specific or s3 product specific questions a general perception is s3 and ebs can be used interchangeably and the interviewer would want to test your knowledge on s3 and evs well ebs uses s3 that's true but they can't be interchangeably used so you might face this question what are some key differences between aws s3 and ebs well the differences are s3 is an object store meaning you can't install anything in it you can store drive files but you can't actually install in it it's not a file system but ebs is a file system you can install services i mean install applications in it and that's going to run stuff like that and talking about performance s3 is much faster and ebs is super faster when accessing from the instance because from the instance if you need to access s3 you'll actually have to go out through the internet and access the s3 or s3 is an external service very external service you'll have to go through or you'll have to go outside of your vpc to access s3 s3 does not come under a vpc but ebs comes under a vpc it's on the same vpc so you would be able to use it kind of locally compared to s3 ebs is very local so that way it's going to be faster and redundancy talking about redundancy of s3 and ebs s3 is replicated the data in s3 is replicated across the data centers but ebs is replicated within the data center meaning s3 is replicated across availability zones ebs is within an availability zone so that way the redundancy is a bit less in ebs in other words redundancy is higher in s3 than eps and talking about security of s3 a s3 can be made private as well as public meaning anybody can access s3 from anywhere in the internet that's possible with s3 but ebs can only only be accessed when attached to an ec2 instance right just one instance can access it whereas s3 is publicly directly accessible the other question related to s3 security is how do you allow access to a user to assert in a user to a certain bucket which means this user is not having access to s3 at all but this user needs to be given access to a certain bucket how do we do it the same case applies to servers as well in few cases there could be an instance where a person is new to the team and you actually don't want them to access the production service now he's in the production group and by default he or she is granted access to that server but you specifically want to deny access to that production server till the time he or she is matured enough to access or understand the process understand the do's and don'ts before they can put their hands on the production server so how do we go about doing it so first we would categorize our instances well these are critical instances these are normal instances and we would actually put a tag on them that's how we categorize right so you put attack on them put attacks saying well they are highly critical they are medium critical and they are not critical at all still there in production stuff like that and then you would pick the users who wants to or who should be or should not be given access to a certain server and you would actually allow the user to access or not access servers based on a specific tag in other words you can use actually tags in in the previous step we put tags on the critical server right so you would define that this user is not going to use this tag all right this user is not allowed to use the resources with this stack so that's how you would make your step forward so you would allow or deny based on the tags that you have put so in this case he or she will not be allowed to servers which are tagged critical servers so that's how you allow deny access to them and the same goes for bucket as well if an organization is excessively using s3 for their data storage because of the benefit that it provides the cost and the durability you might get asked this question which is organizations would replicate the data from one region to another region for additional data durability and for having data redundancy not only for that they would also do that for dr purposes for disaster recovery if the whole region is down you still have the data available somewhere else and you can pick and use it some organizations would store data in different regions for compliance reasons to provide low latency access to their users who are local to that region stuff like that so when companies do replication how do you make sure that there is consistency in the replication how do you make sure that the replication is not failing and the data gets transferred for sure and there are logs for that replication this is something that the companies would use where they're excessively using s3 and they're fully relying on the replication in running their business and the way we could do it is we can set up a replication monitor it's actually a set of tools that we could use together to make sure that the cloud replication a region level replication is happening properly so this is how it happens now on this side on the left hand side we have the region 1 and on the right hand side we have region 2 and region 1 is the source bucket and region 2 is the destination bucket right so object is put in the source bucket and it has to go directly to the region to bucket or or made a copy in the region to bucket and the problem is sometimes it fails and there is no consistency between them so the way you would do it is connect these services together and create and cross replication or cross region replication monitor that actually monitors that actually monitors your environment so there are cloud watts that make sure that the data is uh moved no data is failing again there's cloud watch on the other end make sure that the data is moving and then we have the logs generated through cloudtrail and that's actually written in dynamodb and if there is an error if something is failing you get notified through an sms or you get notified through an email using the sns service so that's how we could leverage these tools and set up and cross region replication monitor that actually monitors your data replication some common issues that company companies face in vpc is that we all know that i can use route 53 to resolve an ip address externally from the internet but by default the servers won't connect to the other servers using our custom dns name that does not do that by default so it's actually a problem there are some additional things that as an administrator or as an architect or as a person who uses it you will have to do and that's what we're gonna discuss so the question could be a vpc is not resolving the server through the dns you can access it through the ip but not through the dns name and what could be the issue and how do you go about fixing it and you will be able to answer this question only if you have done it already it's a quick and simple step by default vpc does not allow that's the default feature and we will have to enable the dns hostname resolution before now this is for the custom dns not for the default dns that comes along this is for the custom dns so we will have to enable the dns host name resolution so our will have to enable dns hostname resolution so they actually resolve let's say i want to connect to a server1.simplylearn.com by default it's not allowed but if i enable this option then i will be able to connect to server1 simplylearn.com if a company has vpcs in different regions and they have a head office in a central place and the rest of them are branch offices and they are connecting to the head office for access or you know for saving data or for accessing certain files or certain data or storing data all right so they would actually mimic the hub and spoke topology where you have the vpc which is centrally in an accessible region a centrally accessible region and then you would have a local vpcs or branch offices in different other regions and they get connected to the vpc in the central location and the question is how do you actually connect the multiple sites to a vpc and make communication happen between them by default it does not do that we know that vpcs they need to be paired between them in order to access the resources let's look at this picture right so i have like a customer network or branch offices in different parts and they get connected to a vpc that's fine so what we have achieved is those different offices the remote offices they are connecting to the vpc and they're talking but they can't connect or they can't talk to each other that's what we have built but the requirement is the traffic needs to or they should be able to talk to each other but they should not have direct connection between them which means that they will have to come and hit the vpc and then reach the other customer network which is in los angeles or which is in new york all right that's the requirement so that's possible with some architecting in the cloud so that's using vpn cloud hub you look at this dotted lines which actually allows customers or which actually allows the corporate networks to talk to each other through the vpc again by default it doesn't happen cloud hub is an architecture that we should be using to make this happen and what's the advantage of it as a central office or as the headquarters office which is in the vpc or headquarters data center which is in the vpc you have control or the vpz has control on who talks to who and what traffic can talk to i mean what traffic can be routed to the other head office stuff like that that centralized control is on the vpc the other question you could get asked is neyman explain some security products and features available in vpc well vpc itself is an security service it provides security service to the application but how do you actually secure the vpc itself that's the question and yes there are products that can actually secure the vpc or the vpc delivers those products to secure the application access to the vpc is restricted through a network access control list right so that's and security product in vpc and a vpc has security groups that protects the instances from unwanted inbound and outbound traffic and network access control list protects the subnets from unwanted inbound and outbound access and there are flow logs we can capture in vpc that captures incoming and outgoing traffic through a vpc which will be used for later analysis as in what's the traffic pattern what's the behavior of the traffic pattern and stuff like that so there are some security products and features available in vpc now how do you monitor a vpc vpc is a very important concept very important service as well everything sits in a vpc most of the service sits in a vpc except for lambda and s3 and dynamodb and couple of other services most of them sit in a vpc for security reason so how do you monitor your vpc how do you gain some visibility on your vpc well we can gain visibility on our vpc using vpc flow log that's the basic service as you see it actually captures what's allowed what's not allowed stuff like that which ip's allowed which ip is not allowed stuff like that so we can gather it and we can use that for analysis and the other one is cloud watch and cloud watch logs the data transfers that happen so this is you know who gets allowed and who does not get allowed i mean the flow logs is who is allowed and who's not allowed that kind of detail and cloud watch gives information about the data transfer how much data is getting transferred we can actually pick unusual data transfers if there is a certain hike in the graph there's a sudden hike and something happens at 12 on a regular basis and you weren't expecting it there's something suspicious it could be valid backups it could be a malicious activity as well so that's how you know by looking at cloudwatch logs and cloudwatch dashboard now let's talk about multiple choice questions when going for an interview you might sometimes find yourself that the company is conducting an online test based on the score they can put you to a panelist and then they would take it forward so we thought we'll also include multiple choice questions to help you better handle such situation if you come across all right when you find yourself in such situation the key to clear them is to understand the question properly read between the lines that's what they say you know there can be like a big paragraph with three lines or ten lines you really got to understand what the question is about and then try to find answer for that question so that's a thumb rule number one and then the second rule is try to compare and contrast the services mentioned or try to compare and contrast the answers you can easily read out one or two answers and then you will be left with only two answers to decide from you know so that also helps you with time and that's all that also helps you with some precision in your answer so number one read between the lines number two compare and contrast the services and you'll be able to easily weed out the wrong ones so let's try answering this question suppose you are a game designer and you want to develop a game with a single digit millisecond latency which of the following database services would you choose so we know that the following are database services are good enough all right and it talks about millisecond latency that's a key point and the third thing is it's a game it could be a mobile game it's a game that you are trying to design and you need the millisecond latency and it has to be a database all right so let's talk about the options available rds rds is a database for sure is it good for a game design we'll come back to that neptune neptune is a graph a database service in amazon so that's kind of out of the equation and snowball is actually a storage right it's it's a transport medium i would say so that's again out of the equation so the tie is between rds and dynamodb if we need to talk about rds rds is an a platform as a service it provides cost efficient resizable capacity but it's an sql database meaning the tables are kind of strict you know it's good for banking and other type of applications but not really good for anything that has to do with the gaming so the only option left is dynamodb again it's the right answer dynamodb is actually an fast and flexible nosql database service and it provides a single digit millisecond latency at any scale and it's a database at the same time it's a key value store model database so the right answer is dynamodb all right let's look at the next question if you need to perform real-time monitoring of aws services and get actionable insights which service would you use all right let's list the services so it talks about real-time monitoring a firewall manager what does it provide now firewall manager is not really a monitor just like the name says it's a manager it manages multiple firewalls and aws guard duty is an threat detection service it does monitoring it does continuously monitor our environment but it monitors for threats all right only threats now let's talk about cloud watch a cloud watch is a service that helps to track metrics it's a service that is used to monitor the environment and give us a system-wide visibility and also it helps us to store logs so at the moment it kind of looks like that could be the right answer we don't know that yet but i mean we have one more option left that's uh ebs so what's ebs ebs is a block storage elastic block store if we abbreviate ebs it's elastic block store so all three of them are easily out of the question the first one is to manage second one is to find threats of course it does monitoring so there's i mean if there is one relation between cloud watch and god duty that's monitoring so easily we can actually find ourselves slipped towards picking guard duty but know that guard duty is only for gaining security inside but not about gaining aws service inside so cloudwatch is a service that helps us to get a system wide or an aws wide or an account wide and it has number of metrics we can monitor and get a very good insight of how a service is performing be it cpu be it ram beat network utilization beat connection failures cloud watch is a service that helps us perform a real-time monitoring and get some actionable insights on the services all right let's talk about this 33rd question as a web developer you are developing an app especially for the mobile platform all right there is a mention that this is especially for the mobile platform so a lot of services gets filtered out mobile platform right which of the following lets you add user sign up sign in and access control to your web and mobile app quickly and easily all right so this is all about signing in to your mobile app so if we need to read between the lines that's how we can read sign up or sign in into an mobile platform all right so we have like four options here uh shield aws massey aws inspector amazon cognito so let's try to weed out services which are not relevant to it so what's aws shield aws shield is actually a service that provides a ddos mitigation or ddos protection denial of service protection it's a security feature let's talk about the second option aws maxi is again a security service that uses machine learning to automatically discover and classify the data it again talks about security and this security is all about encrypting or saving the data does not come close with signing up and mobile platform all right let's talk about the other one aws inspector now aws inspector has something to do with apps it definitely has something to do with apps so kind of looks like that's relevant as of now so it actually helps with improving the security and compliance of the apps that we deploy in the cloud so kind of looks like it could be because it has to do with apps the last one cognito now cognito is a service that actually lets the administrator to have control access over web and mobile apps and it's a service that helps us to sign up and assign in to an mobile and web app so that very much looks like we found the answer so cognito is a service that helps web app and mobile app for sign up and signing in and also gives the administrator to have control over who has access control over the web and the mobile app pretty much we found it so it's cognito cognito is a service that helps us to set up sign up sign in and have access control over the users who would be using our mobile and web app all right how about this question you are an ml engineer or a machine learning engineer who is on the lookout for a solution that will discover sensitive information that your enterprise stores in aws and then uses nlp to classify that data and provide business related insights which among the following services would you choose so we have a bunch of services that's going to help us achieve or one of it is going to help us achieve the about requirement so it's a service that deals with machine learning you're a machine learning engineer who's looking for a service that will help you to discover information at your enterprise store so we're talking about storage discover information in store and then classify the data depending on severity the sensitivity classify the data so which service is that so firewall manager just like the name says it's a manager and the aws iam if we abbreviate it it's a identity and access management so it's identity and access management nothing to do with identifying sensitive data and managing it so the first two is already out of the equation then the aw is messy we already had a quick definition description for aws massey that it's actually a security service that uses machine learning kind of looks like it could be it it's a security service that uses machine learning and it discovers and classifies the sensitive information not only that it does not stop there it goes beyond and protects the sensitive data aws massey kind of looks like but we still have one more option to look at which is cloud hms cloud hms is also a security service kind of looks like that could be the answer as well and it enables us to generate encryption keys and save the data so kind of fifty percent of it it's a security service it encrypts helps us protect the data but aws maxi is right on spot it's a machine learning service it helps us to classify the data and also to protect the data so the answer for this question would be aws messy so hope you kind of get it how this is going so first we apply the thumb rule identify the question that's being asked read between the lines and then try to find the service that meets your requirement then finding the servers is by first weeding out the wrong ones recollect everything that you've learned about the service and see how well that matches with those hints that you have picked up and if that doesn't match weed that out then you'll end up with two just to to decide from at some point and then it becomes easy for you to decide click on the question submit it and then move on to the other question in your interview alright so how about this one you are a system administrator in a company which is running most of its infrastructure on aws you are required to track your users and keep a look on how your users are being authenticated all right so this is where the problem statement starts right you need to keep track of how your users are being authenticated and you wish to create and manage aws users and use permissions to allow and deny their access to the aws resources right so you are to give them permission number one and then i mean if we put them in the right order first giving them permissions and then tracking their usage let's see which of the service will help us achieve it iam is a service that helps us to looking at the permissions we can actually predict whether the user or the group will have servers or not so that helps us to get a track of who is able to use who's not able to use certain servers and all that stuff so it kind of looks like but we have other three options left let's look at aws firewall manager just like the name says it's actually a firewall manager it helps us to manage multiple firewalls simple as that and shield is a service it's a service that's used to protect denial of service or distributed denial of service an api gateway is a service that makes it easy for developers to create publish maintain and monitor and secure api so i mean it's completely on the api side very less on user and how you authenticate your user we can get that by looking at the name itself right if you abbreviate it or if you if you try to find a definition for the name api gateway you would get it it has to do with api but if you aggregate aws iam its identity and access management pretty much meets the requirement for the problem statement about its aws identity and access management that's the right answer all right let's look at this one if you want to allocate various private and public ip address in order to make them communicate with the internet and other instances you will use this service which of the following is this service so it talks about using public and private ip address so this service uses ip address and then this service helps us to allow and deny connections to the internet and to the other instances so you get the question is it let's pick the service that helps us achieve it route 53 route 53 is actually a dns service right so it's not a service that's used to allow or deny no it does not do that vpc vpc uses public and private ip address yes so kind of looks like a vpc helps us to allow i mean the security in vpc the security group the network access control list in a vpc that outing table in a vpc that actually helps us to allow or deny a connection to a particular ip address or to a particular service within the vpc or outside of the vpc so as of now it kind of looks like it could be but let's look at the other services what if if we find a service that closely matches to the above requirement than the amazon vpc gateway api gateway we know that it's a managed service that makes it easy for developers to create publish maintain and monitor apis and secure api so that has completely do with api not with ip cloudfront we know about cloudfront that it's a content delivery network and it provides global distribution of servers where our content can be cached it could be video or or bulk media or anything else they can be cached locally so users can easily access them and download them easily right so that's cloud front now at this point after looking at all four it looks like vpc is the right answer and in fact vpc is the right answer vpc has public ip address vpc can help us with a private ip address vpc can be used to allow deny connection based on the security group access control list and the routing table it has so that's right answer is vpc all right how about this one this platform as a service or platform as a db service provides us with the cost efficient and resizable capacity while automating time consuming administrative tasks so this question is very clear it's a db service we got to look for and it's a service that can provide automating some of the time consuming tasks it has to be resizable at the same time so let's talk about amazon rational database it's a database kind of matches the requirement we can resize it as and when needed all right looks like it's a fit as of now it actually automates some of the time consuming work looks like it's a fit as of now let's move on to elastic cache and then try to see if that matches the definition that we've figured out about elastic cache it's actually a caching service it's again an in-memory data store which helps in achieving high throughput and low latency in memory data store so it's not a full-blown database and it does not come with any amazon provisioned automation in it for automating any of the administration tasks no it does not come up with anything like that yeah we can resize the capacity as in when needed but automation it's not there yet and moreover it's not a database so that's out of the equation vpc is not a recessible one you know once we have designed vpc it's fixed it can't be resized so that's out of the equation and uh amazon glacier glacier is a storage but not a database all right so that's again of the equation so that tie is kind of between amazon rational database service and amazon elastic cache because they both aid the database service but elastic cache is not a full-blown database it actually helps database but it's not a full-blown database so it's amazon relational database that's the one which is a platform as a service it's the one which can be resized it's the one which can be used to automate the time consuming administrative tasks all right let's talk about this one uh which of the following is a means for accessing human researchers or consultants to help solve a problem on a contractual or a temporary basis all right let's read the question again which of the following is a means for accessing human researchers or consultant to help solve problems on a contractual or a temporary basis it's like a signing task or hiring aws experts for a temporary job so let's try to find that kind of service in the four services that are listed amazon elastic mapreduce mapreduce is actually an framework service that makes it easy and cost effective to analyze large amount of data but that has nothing to do with accessing human researchers all right let's talk about mechanical term it's a web service that provides a human workforce that's the definition for it for example automation is good but not everything can be automated for something to qualify for automation it has to be and repeated tasks one time task can't be automated or the time and money that you would be spending in automation is not worth it instead you could have done it manually so that does not qualify for automation and anything that requires intelligence right anything that's a special case right automation can do repetitive tasks automation can do precise work but it has to be repeated tasks the scenario you know it should have been there already only then that can be executed but if it's a new scenario and it requires uh appropriate addressing then it requires human thought so we could hire researchers and consultants who can help solve a problem using amazon mechanical turk the other two are already out of the equation now dev pay is actually a payment system through amazon and multi-factor authentication as it says it's an authentication system so the right answer is amazon mechanical turk all right this sounds interesting let's look at this one this service is used to make it easy to deploy manage and scale containerized applications using kubernetes on aws which of the following is this aws service so it's a service to deploy manage and scale containerized applications so it deals with containers it also should have the ability to use kubernetes which is an container orchestration service all right the first one amazon elastic container service kind of looks like it's the one the name itself has the word and the relation we're looking for elastic container service so this container service is an highly scalable high performance container orchestration service let's look at the other one aws batch it's a service that enables id professionals to schedule and execute batch processing i mean the name itself says that that's meant for batch processing elastic bean stock that's another service that helps us to deploy manage and scale but it helps us with ec2 instances not with containerized instances so that's again out of the equation would light scale be a good time for elastic container service what's light sale now light sale is a service it's called as a virtual private server without a vpc it's called as a virtual private server it comes with a predefined compute storage networking capacity it's actually a server not a container right so at this point that also becomes out of the equation so it's amazon elastic container service that's the one that helps us to easily deploy manage scale container services and it helps us orchestrate the containers using kubernetes all right how about this one all right this service lets us to run code without provisioning or managing servers so no servers run code select the correct service from the below option all right so no servers but we should be able to run code amazon easy to order scaling easy to auto scaling ec2 is elastic compute cloud which is a server and auto scaling is a service that helps us to achieve scaling the server so that's the definition for it could be that's out of the equation aw is lambda now lambda is a service it's actually an event driven serverless computing platform and lambda runs code in response to the event that it receives and it automatically manages the compute resource that's required for that code as long as we have uploaded a code that's correct and set up events correctly to map to that code it's going to run seamlessly so that's about lambda it kind of looks like it could be the answer because lambda runs code we don't have to manage servers it manages servers by itself but we can't conclude as of now we have other two servers to talk about aws batch all right batch is service that enables id professionals to run batch job we know that and about inspector amazon inspector it's actually a service that helps us to increase and identify any security issues and align our application with compliance well that's not the requirement of the question the requirement in the question was run code without provisioning your server and without any more space for confusion aws lambda is a service or is the service that runs code without provisioning and managing services right the right one would be aws lambda i'm very excited that you're watching this video and i'm equally glad that uh we were able to provide you a second part in aws interview questions all right let's get started so in an environment where there's a lot of automation infrastructure automation you'll be posted with this question how can you add an existing instance to a new auto scaling group now this is when you are taking an instance away from the auto scaling group to troubleshoot to fix a problem you know to look at logs or if you have suspended the auto scaling you know you might need to re-add that instance to the auto scaling group only then it's going to take part in it right only then the auto scaling is going to count it as part of it it's not a straight uh procedure you know when you remove them you know it doesn't get automatically re-added i've had worked with some clients so when their developers were managing their own environment they had problems adding the instance back to the auto scaling group you know irrespective of what they tried the instance was not getting added to the auto scaling group and whatever they fixed that they were provided or whatever fix that they have provided we're not you know getting encountered in the auto scaling group so like i said it's not a straight you know a click button procedure there are ways we'll have to do it so how can you add an existing instance uh to the auto scaling group there are a few steps that we need to follow so the first one would be to under the ec2 instance console right under the uh instance under actions in specific you know there's an option called attach to auto skilling group right if you have multiple auto scaling groups in your account or in the region that you're working in then you're going to be posted with the different auto scaling groups that you have in your account let's say you have five auto scaling groups for five different application you know you're going to be posted with five different auto scaling groups and then you would select the auto scaling the appropriate auto scaling group and attach the instance to that particular auto scaling group while adding to the auto scaling group if you want to change the instance type you know that's possible as well sometimes when you want to add the instance back to the auto skilling group there would be requirement that you change the instance type to a better one to a better family to the better instance type you could do that at that time and after that you are or you have completely added the instance back to the auto scaling group so it's actually an seven step uh process adding an instance back to the auto scaling group in an environment where they're dealing with migrating the instance or migrating an application or migrating an instance migrating and vm into the cloud you know if the project that you're going to work with deals with a lot of migrations you could be posted this question what are the factors you will consider while migrating to amazon web services the first one is cost is it worth a moving the instance to the cloud given the additional bills and vessels features available in the cloud is this application going to use all of them is moving into the cloud beneficial to the application in the first place you know beneficial to the users who will be using the application in the first place so that's a factor to think of so this actually includes no cost of the infrastructure and the ability to match the demand and supply transparency is this application and high demand you know is it going to be a big loss if the application becomes unavailable for some time so there are a few things that needs to be considered before we move the application to the cloud and then if the application does the application needs to be provisioned immediately is there an urge is there an urge to provision the application immediately that's something that needs to be considered if the application requires to go online if the application needs to hit the market immediately then we would need to move it to the cloud because in on premises procuring buying an infrastructure buying the bandwidth buying the switchboard you know buying an instance you know buying their softwares buying the license related to it it's going to take time at least like two weeks or so before you can bring up an server and launch an application in it right so the application cannot wait you know waiting means uh you know workforce productivity loss is it so we would want to immediately launch instances and put application on top of it in those case if your application is of that type if there is urge in making the application go online as soon as possible then that's a candidate for moving to the cloud and if the application or if the the software or if the product that you're launching it requires hardware it requires an updated hardware all the time that's not going to be possible in on-premises we try to deal with legacy infrastructure all the time in on-premises but in the cloud they're constantly upgrading their hardwares only then they can keep themselves up going in the market so they constantly the cloud providers are constantly updating their hardwares and if you want to be benefited of your application wants to be benefited by the constant upgrading of the hardwares making sure the hardware is as latest as possible the software version the licensing is as latest as possible then that's a candidate to be moved to the cloud and if the application does not want to go through any risk if the application is very sensitive to failures if the application is very much stacked to the revenue of the company and you don't want to take a chance in you know seeing the application fail and you know seeing the revenue drop then that's a candidate for moving to the cloud and business agility you know moving to the cloud at least half of the responsibility is now taken care by the provider in this case it's amazon at least half of the responsibility is taken care by them like if the hardware fails amazon makes sure that they're fixing the hardware immediately and notifications you know if something happens you know there are immediate notifications available that we can set it up and make yourself aware that something has broken and we can immediately jump in and fix it so you see there are the responsibility is now being shared between amazon and us so if you want to get that benefit for your application for your organization for the product that you're launching then it needs to be moved to the cloud so you can get that benefit from the cloud the other question you could get asked is what is rto and rpo in aws they are essentially disaster recovery terms when you're planning for disaster recovery you cannot avoid planning disaster recovery without talking about rto and rpo now what's the rto what's the rpo in your environment or how do you define rto how do you define rpo or some general questions that get asked rto is recovery time objective rto stands for the maximum time the company is willing to wait for the recovery to happen or for the recovery to finish when and disaster strikes so rto is uh in the future right how much time is it going to take to fix and bring everything to normal so that's rto on the other hand rpo is recovery point objective which is the maximum amount of data laws your company is willing to accept as measured in time rpo always refers to the backups the number of backups the the frequency of the backups right because when an outage happens you can always go back to the latest backup right and if the latest backup was data storage right so rpo is the acceptable amount if the company wants less rpo rpo is 1r then you should be planning on taking backups every one hour if rpo is 12 hours then you should be planning on taking backups every 12 hours so that's how rpo and rto you know helps disaster recovery the fourth question you could get asked is if you would like to transfer huge amount of data which is the best option among snowball snowball edge and snowmobile again this is a question that get asked if the company is dealing with a lot of data transfer into the cloud or if the company is dealing with the migrating data into the cloud i'm talking about a huge amount of data data in petabytes snowball and all of the snowball series deals with the petabyte-sized data migrations so there are three options available as of now aws snowball is an data transport solution for moving high volume of data into and out of a specified aws region on the other hand aws snowball edge adds additional computing functions snowball is simple storage and movement of data and snowball edge has a compute function attached to it snow mobile on the other hand is an exabyte scale migration service that allows us to transfer data up to 100 petabytes that's like 100 000 terabytes so depending on the size of data that we want to transfer from our data center to the cloud we can hire we can rent any of these three services let's talk about some cloud formation questions this is a classic question how is aws cloud formation different from aws elastic bean stock you know from the surface they both look like the same you know you don't go through the console provisioning resources you don't you know you don't go through cli and provision resources both of them provision resources through click button right but underneath they are actually different services they support they aid different services so knowing that is going to help you understand this question a lot better let's talk about the difference between them and this is what you will be explaining to the interviewer or the panelist so the cloud formation the cloud formation service helps you describe and provision all the infrastructure resources in the cloud environment on the other hand elastic bean stock provides an simple environment to which we can deploy and run application cloud formation gives us an infrastructure and elastic beanstalk gives us an small contained environment in which we can run our application and the cloud formation supports the infrastructure needs of many different types of application like the enterprise application the legacy applications and any new modern application that you want to have in the cloud on the other hand the elastic bean stock it's a combination of developer tools they are tools that helps manage the life cycle of a single application so cloud formation in short is managing the infrastructure as a whole and elastic bean stock in short is managing and running an application in the cloud and if the company that you're getting hired is using cloud formation to manage their infrastructure using or if they're using infrastructure or any of the infrastructure as a code services then you would definitely face this question what are the elements of an aws cloud formation template so it has four or five basic elements right and the template is in the form of json or in yaml format right so it has parameters it has outputs it has data it has resources and then the uh format or the format version or the file format version for the cloud formation template so parameter is nothing but it actually lets you to specify the type of ec2 instance that you want the type of rds that you want all right so ec2 is an umbrella rds is an umbrella and parameters within that ec2 and parameters within that rds are the specific details of the ec2 or the specific details of the rds service so that's what parameters in a cloud formation template and then the element of the cloud formation template is outputs for example if you want to output the name of an s3 bucket that was created if you want to output the name of the ec2 instance if you want to output the name of some resources that have been created instead of looking into the template instead of you know navigating through in the console and finding the name of the resource we can actually have them outputted in the result section so we can simply go and look at all the resources created through the template in the output section and that's what output values or output does in the cloud formation template and then we have resources resources are nothing but what defines what are the cloud components or cloud resources that will be created through this cloud formation template now ec2 is a resource rds is a resource and s3 bucket is a resource elastic load balancer is a resource and nat gateway is a resource vpc is a reserve so you see all these components are the resources and the resource section in the cloud formation defines what are the aws cloud resources that will be created through this cloud formation template and then we have version a version actually identifies the capabilities of the template you know we just need to make sure that it is of the latest version type and the latest version is 0 9 0 9 that's the latest version number you'll be able to find that on the top of the cloudformation template and that version number defines the capabilities of the cloudformation template so just need to make sure that it's the latest all the time still talking about cloud formation this is another classic question what happens when one of the resource in a stack cannot be created successfully well if the resource in a stack cannot be created the cloud formation automatically rolls back and dominates all the resources that was created using the cloud formation template so whatever resources that were created through the cloud formation template from the beginning let's say we have created like 10 resources and the 11th resource is now failing cloud formation will roll back and delete all the 10 resources that were created previously and this is very useful uh when the cloud formation cannot you know go forward cloud formation cannot create additional resources because we have reached the elastic ip limits elastic ip limit per region is five right and if you have already used five ips and a cloud formation is trying to buy three more ips you know we've hit the soft limit till we fix that with amazon cloudformation will not be able to you know launch additional you know resources and additional ips so it's going to cancel and roll back everything that's true with a missing ec2 ami as well if an ami is included in the template and but the ami is not actually present then cloud formation is going to search for the mi and because it's not present it's going to roll back and delete all the resources that it created so that's what cloud formation does it simply rolls back all the resources that it created i mean if it sees a failure it would simply roll back all the resources that it created and this feature actually simplifies the system administration and layout solutions built on top of aws cloud formation so at any point we know that there are no orphan resources in the in in our environment you know because something did not work or because there was an you know cloud formation executed some there are no orphan resources in our account at any point we can be sure that if cloud formation is launching a resource and if it's going to fail and it's going to come back and delete all the resources it's created so there are no orphan resources in our account now let's talk about some questions in elastic block store again if the environment deals with a lot of automation you could be thrown this question how can you automate easy to backup using ebs it's actually a six step process to automate the ec2 backups we'll need to write a script to automate the below steps using aws api and these are the steps that should be found in the scripts first to get the list of instances and then and then the script that we are writing should be able to connect to aws using the api and list the amazon abs volumes that are attached locally to the instance and then it needs to list the snapshots of each volume make sure the snapshots are present and it needs to assign a retention period for the snapshot because over time the snapshots are going to be invalid right once you have some 10 latest snapshots any snapshot that you have taken before that 10 becomes invalid because you have captured the latest and 10 snapshot coverage is enough for you and then the fifth point is to create a snapshot of each volume create a new snapshot of each volume and then delete the old snapshot anytime a new snapshot gets created the oldest snapshot of the list needs to go away so we need to include options we need to include scripts in our script lines in our script that make sure that it's deleting the older snapshots which are older than the retention period that we are mentioning another question that you could see in the interview be it a written interview beat online interview or beat and telephonic or face-to-face interview is what's the difference between ebs and instant store let's talk about ebs first ebs is kind of permanent storage the data in it can be restored at a later point when we save data in ebs the data lives even after the lifetime of the ec2 instance for example we can stop the instance and the data is still going to be present in ebs we can move the ebs from one instance to another instance and the data is simply going to be present there so abs is kind of permanent storage when compared to instance on the other hand instance store is a temporary storage and that storage is actually physically attached to the host of the machine ebs is an external storage an instant store is locally attached to the instance or locally attached to the host of the machine we cannot detach an instant store from one instance and attach it to another but we can do that with eba so that's a big difference one is permanent data and another one is ebs is permanent instant store is a volatile data and um instant store with instant store we won't be able to detach the storage and attach it to another instance and another feature of instant store is data in an instant store is lost if the disk fails or the instance is stopped or terminated so instant store is only good for storing cache data if you want to store permanent data then we should think of using ebs and not instant store while talking about storage on the same lines this is another classic question how can you take backups of efs like ebs and if you can take backup how do you take that backup the answer is yes we can take efs to efs backup solution efs does not support snapshot like ebs efs does not support snapshot snapshot is not an option for efs elastic file system right we can only take backup from one efs to another efs and this backup solution is to recover from unintended changes or deletions of the efs and this can be automated but any data that we store in efs can be automatically replicated to another efs and once this efs goes down or gets deleted or data gets deleted or you know the whole ef is is for some reason interrupted or deleted we can recover the data from we can use the other efs and bring the application to consistency and to achieve this it's not an one step configuration it's a cycle there are series of steps that's involved before we can achieve efs to efs backup the first thing is to sign in to the aws management console and under efs or click on efs to efs restore button from the services list and from there we can use the region selector in the console navigation bar to select the actual region in which we want to work on and from there ensure that we have selected the right template you know some of the templates would be you know efs to efs backup granular backups incremental backups right so there are some templates the kind of backups that you want to take do you want to take grant alert do you want to take increment backups stuff like that and then create a name to that solution the kind of backup that we have created and finally review all the configurations that you have done and click on save and from that point onwards the data is going to be copied and from that point onwards any additional data that you put is going to copy it and replicate it now you have an efs to efs backup this is another classic question in companies which deals with a data management there are easy options to create snapshots but deleting snapshots is not always an you know click button or an single step configuration so you might be facing a question like how do you auto delete old snapshots and the procedure is like this now as best practice we will take snapshots of ebs volume to s3 all snapshots get stored in s3 we know that now and we can use aws ops automator to automatically handle all snapshots the ops automator service it allows us to create copy delete ebs snapshots so there are cloud formation templates available for aws ops automator and this automator template will scan the environment and it would take snapshots it would you know copy the snapshot from one region to another region if you want i know if you're setting up a dr environment and not only that based on the retention period that we create it's going to delete the snapshots which are older than the retention period so life or managing snapshot is made a lot easier because of this ops automator cloud formation template moving into questions and elastic load balancer this again could be an a question in the interview what are the different types of load balancers in aws and what's their use case what's the difference between them and as of now as we speak there are three types of load balancers which are available in aws the first one being application load balancer just like the name says the application load balancer works on the application layer and deals with the http and https request and it also supports part based routing for example simplylearn.com slash some webpage simplylearn.com slash another website so it's going to direct the path based on the slash value that you give in the urls that's path based routing so it supports that and not only that it can support a port based port-based colon 8080 colon 8081 or colon 8090 you know based on that port also it can take a routing decision and that's what application load balancer does on the other hand we have network load balancer and the network load balancer makes routing decisions at the transport level it's faster because it has very less thing to work on it works on lower osi layer it works on a lower layer so it has very less information to work with than compared with application layers so comparatively it's a lot faster and it handles millions of requests per second and after the load balancer receives the connection it selects a target group for the default rule using the flow hash routing algorithm it does simple routing right it does not do path-based or port-based routing it does simple routing and because of it it's faster and then we have classic load balancer which is kind of expiring as we speak amazon is discouraging people using classic load balancer but there are companies which are still using classic load balancer they are the ones who are the first one to step into amazon when classic load balancer was the first load balancer or the only load balancer available at that point so it supports http https tcp ssl protocol and it has a fixed relationship between a load balance report and the container port so initially we only have classic load balancer and then at after some point amazon said instead of having one load balancer address all type of traffic we're going to have two load balances called as the child from the classic two load balancer and one is going to specifically address the application requirement and one is going to specifically address the network requirement and let's call it as application load balancer and network load balancer so that's how now we have two different load balancers talking about load balancer another classic question could be what are the different uses of the various load balancer in aws elastic load balancing there are three types of load balancer we just spoke about it application load balancer is used if we need a flexible application management and a tls termination and network load balancer if we require extreme performance and the load balancing should happen on based on static ips for the application and classic load balancer is an old load balancer which is for people who are still running their environment from ec2 classic network now this is an older version of vpc or this is what was present before vpc was created easy to classic network is what was present before ec2 was created so they are the three types and they are the use cases of it let's talk about some of these security related questions you would face in the interview when talking about security and firewall in aws we cannot avoid discussion talking about waff web application firewall and you would definitely see yourself in this situation where you've been asked how can you use aws off in monitoring your aws applications waf or web application firewall protects our web application from common web exploits and vaf helps us control which traffic source your application should be allowed or a block which traffic from a certain source you know which source or which traffic from a certain source should be allowed or blocked your application with waff we can also create custom rules that blocks common attack patterns you know if it is a banking application it has a certain type of attacks and if it is simple uh data management data storage application it has i mean content management application it has a separate type of attack so based on the application type we can identify a pattern and create rules that would actually block that attack based on the rule that we create and waff can be used for three cases you know the first one is allow all requests and then a block all request and count all requests for a new policy so it's also an monitoring and management service which actually counts all the policies or counts all the requests that matches a particular policy that we create and some of the characteristics we can mention in awsof are the origin ips and the strings that appear in the request we can allow block based on origin ip allow block based on strings that appear in the request we can allow block or count based on the origin country length of the request yeah we can block and count the presence of malicious scripts in an connection now we can count the request headers or we can allow block a certain request header and we can count the presence of a malicious sql code in a connection that we get and that want to reach our application still talking about security what are the different aws im categories we can control using aws iam we can do the following one is create and manage iem users and once the user database gets bigger and bigger we can create and manage them in groups and in im we can use it to manage the security credentials kind of setting the complexity of the password you know setting additional authentications you know like mfa and you know rotating the passwords you know resetting the password there are a few things we could do with iam and finally we can create policies that actually grants access to aw services and resources another question you will see is what are the policies that you can set for your users password so some of the policies that we can set for the user password is at the minimum length or you know the complexity of the password by at least having one number or one special characters in the password so that's one and then the requirement of a specific character types including you know uppercase lowercase number and non-alphabetic characters so it becomes very hard for somebody else to guess what the password would be and and try to hack them so we can set the length of the password we can set the complexity in the password and then we can set an automatic expiration of the password so after a certain time the user is forced to create a new password so the password is not stale old and easy to guess in the environment and we can also set settings like the user should contact the admin i mean when the password is about to expire so you know you can get a hold of how the user is setting their password is it having good complexity in it is it meeting company standards or there are a few things that we can control and set for the users when the users are setting or recreating the password another question that could be posted in an interview so to understand your understanding of iem is what's the difference between an iem role and an im user let's talk about iem user let's start small and then go big or let's start simple and then talk about the complex one the iem user has a permanent long-term credential and it's used to interact directly with aws services and on the other hand iem role is an iem entity that defines a set of permissions for making aws service request so iem user is an permanent credential and role are temporary credentials and iem user has full access to all aws iem functionalities and with role trusted entities such as iem users application or aws services uh assume of the role so when an iem user is given an permission you know it sticks within the iem user but with roles we can give permissions to applications we can give permissions to users in the same account in a different account a corporate id we can give permissions to ec2 s3 rds vpc and lot more role is wide and im user is is not so wide you know it's very constrained only for that im user let's talk about managed policies in aws managed policies there are two types you know customer managed and amazon managed so manage policies are im resources that express permissions using the iam policy language we can create policies edit them manage them manage them separately from the im user group and roles which they are attached to so they are something that we can do to managed policies if it is customer managed and we can now update policy in one place and the permissions automatically extend to all the attached entries so i can have like three services four services point to a particular policy and if i edit that particular policy it's gonna reflect on those three or four services so anything that i allow is going to be allowed for those four services anything that i denied is going to be denied for uh the four services imagine what would be without the i am managed policy will have to go and specifically allow deny on those different instances four or five times depending on the number of instances that we have so like i said there are two types of managed policies one is managed by us which is customer managed policies and then the other is managed by aws which is aws managed policy this question can you give an example of an iem policy and a policy summary this is actually to test how well versed are you with the aws console the answer to that question is look at the following policy this policy is used to grant access to add update and delete objects from a specific folder now in this case name of the folder is example folder and it's present in a bucket called example bucket so this is an iam policy on the other hand the policy summary is a list of access level resource and conditions for each service defined in a policy so im policy is all about one particular resource and the policy summary is all about multiple resources with iam policy it was only talking about s3 bucket and one particular s3 bucket here it talks about cloud formation template cloud watch logs ec2 elastic bean stock services summary summary of resources and the permissions and policies attached to them that's what policy summary is all about another question could be like this what's the use case of iam and how does im help your business two important or primary work of im is to help us manage iam users and their access it provides a secure access to multiple users uh to their appropriate aws resources so that's one it does and the second thing it does is manage access for federated users federated users or non iam users and through iam we can actually allow and provide a secure access to resources in our aws account to our employees without the im user no they could be authenticated using the active directory they could be authenticated using the facebook credential google credential amazon credential and a couple of other credentials third party identity management right so we could actually trust them and we could give them access to our account uh based on the trust relationship that we have built with the other identity of systems right so two things one is manage users and their access for manage iam user and their access in our aws environment and second is manage access for federated users who are non-iam users and more importantly im is a free service and with that will only be charged for the use of the resources not for the im username and password that we create all right let's now talk about some of the questions in route 53 one classic question that could be asked in an interview is what is the difference between latency based routing and geodns or jio based dns routing now the jio based dns routing takes routing decisions on the basis of the geographic location of the request and on the other hand the latency based routing utilizes latency measurements between networks and data centers now latency based routing is used where you want to give your customers the lowest latency as possible so that's when we would use latency based routing and on the other hand the geo-based routing is when we want to direct customers to different websites based on the country they are browsing from you know you could have you know two different or three different websites for the same url you know take amazon the shopping website for example when we go to amazon.com from in the us it directs us to the u.s web page where the products are different the currency is different right and the flag and a couple of other advertisements that shows up are different and when we go to amazon.com from india it gets directed to the amazon.com indian site where again the currency the product and the advertisements they're all different right so depending on the country they're trying to browse if you want to direct customers to two or three different websites we would use a jio based routing another use case of geo based routing is if you have a compliance that you should handle all the dns requests sorry if you should handle all the requests you know from a country within the country then you would do geo based routing now you wouldn't direct the customer to a server which is in another country right you would direct the customer to a server which is very local to them that's another use case of jio based routing and like i said for latency based routing the whole goal or aim is to achieve minimum end user latency if you are hired for the architect role and if that requires working lot on the dns then you could be posted with this question what is the difference between domain and a hosted zone a domain is actually a collection of data describing self-contained administrative and technical unit on the internet right so for example you know simplylearn.com is actually a domain on the other hand hosted zone is actually an container that holds information about how you want to route traffic on the internet to a specific domain for example lms.simplylearn.com is an hosted zone whereas simplylearn.com is a domain so in other words in hosted zone you would see the domain name plus and a prefix to it lms is a prefix here ftp is a prefix mail.simplylearn.com is a prefix so that's how you would see a prefix in hosted zones another question from another classic question from route 53 would be how does amazon drop 53 provide high availability and low latency the way amazon ralph 53 provides high availability and low latency is by globally distributed dns servers amazon is a global service and they have dna services globally any customer doing a query from different parts of the world they get to reach an dns server which is very local to them and that's how it provides low latency now this is not true with all the dns providers there are dns providers who are very local to a country who are very local to a continent so they don't they generally don't provide low latency service right it's always high latency it's low latency for local users but anybody browsing from a different country or a different continent it's going to be high latency for them but that's not again true with amazon amazon is a globally distributed dns provider it has dns servers global wide and like i said it has optimal locations it has got global servers or in other words it has got servers around the globe different parts in the globe and that's how they are able to provide high availability and because it's not running on just one server but on many servers they provide high availability and low latency if the environment that you're going to work on is going to take a lot of configuration backups environmental backups then you can expect questions in aws config a classic question would be how does aws config work along with aws cloudtrail aws cloudtrail actually records user api activity on the account and you know any http https access or any any sort of access you know that's made to the cloud environment that's recorded in the cloud trail in other words any api calls the time is recorded the type of call is recorded and what was the response given was it a failure was it successful they also get recorded in cloud trail it's actually a log it actually records the activity in your cloud environment on the other hand config is an uh point in time configuration details of your resources for example at a given point what are all the resources that were present in my environment what are all the resources or what are the configuration in those resources at a given point they get captured in aws config right so with that information you can always answer the question what did my aws resource look like at a given point in time that question gets answered when we use aws config on the other hand with cloudtrail you can answer the question i mean by looking at the cloud trail or with the help of cloudtrail you can easily answer the question who made an apa call to modify this resource that's answered by cloudtrail and with the cloud trail we can detect if a security group was incorrectly configured and who did that configuration let's say there happened to be a downtime and you want to identify let's say there happen to be a downtime and you want to identify who made that uh change in the environment you can simply look at cloudtrail and find out who made the change and if you want to look at how the environment looks like before the change you can always look at aws config can aws configure or aws config aggregate data across different aws accounts yes it can now this question is actually to test whether you have used aws config or not i know some of the services are very local is it some of these services are availability zone specific some of them are regional specific and some of them are global services and amazon and though some of the services are region services you still can do some changes you know add some configuration to it and collect regional data in it for example s3 is a regional service but still you can collect logs from all other regions into an s3 bucket in one particular region that's possible and cloud trail is and cloud watch is an regional service but still you can with some changes to it with some adding permissions to it you can always monitor the cloud watch that belongs to cloudwatch logs that belongs to other regions you know they're not global by default but you can do some changes and make it global similarly aws config is a service that's a region based service but still you can make it act globally you can aggregate data across a different region and different accounts in an aws config and deliver the updates from different accounts to one s3 bucket and can access it from there aws config also works or integrates seamlessly with sns topic so you know anytime there is a change anytime a new data gets collected you can always notify yourself or notify a group of people about the new log or the new config or new edit that happened in the environment let's look at some of the database questions you know database should be running on reserved instances so whether you know that fact or not the interviewer wants to understand how well you know that fact by asking this question how are reserved instances different from on-demand db instances reserved instances and on-demand instances are exactly the same when it comes to their function but they only differ based on how they are built reserved instances are purchased for one year or three year reservation and in return we get a very low per hour pricing because we're paying upfront it's generally said that reserved instance is 75 percentage cheaper than on-demand instance and amazon gives you that benefit because you know you're committing for one year and sometimes you're paying in advance for the whole year on the other hand on-demand instances are built on an oddly early price talking about auto scaling how will you understand the different types of auto scaling the interviewer might ask this question which type of scaling would you recommend for rds and y the two types of scaling as you would know now are vertical and horizontal and in vertical scaling we can vertically scale up the master database with a couple of clicks all right so that's vertical scaling vertical scaling is keeping the same node and making it bigger and bigger if previously it was running on t2 micro now we would like to run it on m3 two times large instance previously it had one virtual cpu one gigabit now it's going to have eight virtual cpu and 30 gigabit of ram so that's vertical scaling on the other hand horizontal scaling is adding more nodes to it previously it was running on one vm now it's going to run on 2 3 10 vms right that's horizontal scaling so database can only be scaled vertically and there are 18 different types of instances we can resize our rds to right so this is true for rds mysql postgres sql mariadb oracle microsoft sql servers there are 18 type of instances we can vertically scale up to on the other hand horizontal scaling are good for replicas so they are read only replicas we're not going to touch the master database we're not going to touch the primary database but i can do horizontal scaling only with amazon aurora and i can add additional read replicas i can add up to 15 uh read replicas for amazon aurora and up to five read replicas for rds mysql postgres sql and marie db rds instances and when we add replica we are horizontally scaling adding more nodes right read-only nodes so that's horizontal scaling so how do you really decide between vertical and horizontal scaling if you're looking in to increase the storage and the processing capacity we'll have to do a vertical scaling if you're looking at increasing the performance or of the read heavy database we need to be looking for horizontal scaling or we need to be implementing horizontal scaling in our environment still talking about database this is another good question you can expect in the interview what is the maintenance window in amazon rds will your db instance be available during the maintenance event all right so this is really to test how well you have understood the sla how well you have understood the amazon rdas uh the failover mechanism of amazon rdas uh stuff like that so audio's maintenance window it lets you decide when adb instance modification a database engine upgrades or software patching has to occur and you you actually get to decide should it happen at 12 in the night or should it happen at afternoon should it happen early in the morning should it happen in the evening you actually get to decide an automatic scheduling by amazon is done only for patches that are security and durability related sometimes amazon takes down and does automatic scheduling if you know if there is a need for a patch update that deals with security and durability and by default the maintenance window is is for 30 minutes and the important point is the db instance will be available during that event because you're going to have primary and secondary write so when that upgrade happens amazon would shift the connection to the secondary do the upgrade and then switch back to the primary another classic question would be what are the consistency models in dynamodb in dynamodb there is eventual consistency read this eventual consistency model it actually maximizes your read throughput and the best part with eventual consistency is all copies of data reach uh consistency within a second and sometimes when you write and when you're you know trying to read immediately chances that you would still be reading the old data that's eventual consistency on the other hand there is another consistency model called the strong consistency or strongly consistent lead where there is going to be a delay in writing the data you know making sure the data is written in all places but it guarantees one thing that is once you have done a write and then you're trying to do a read it's going to make sure that it's going to show you the updated data not the old data now you can be guaranteed of it that it is going to show the updated data and not the old data that's strongly consistent still talking about database talking about nosql dynamodb or nosql database which is dynamodb and amazon you could be asked this question what kind of query functionality does dynamodb support dynamodb supports get and put operation dynamodb supports or dynamodb provides flexible querying by letting you query on non-primary key attributes using global secondary index and local secondary indexes a primary key can be either a single attribute partition key or a composite partition sort key in other words a dynamodb indexes a composite partition sort key as a partition key element and a sort key element and by holding the partition key you know when doing a search or when doing a query by holding the partition key element constant we can search across the sort key element to retrieve the other items in that table and the composite partition sort key should be a combination of user id partition and a timestamp so that's what the composite partition sort key is made of let's look at some of the multiple choice questions you know sometimes some companies would have an uh written test or an mcq type online test before they call you for at the first level or before they call you for the second level so these are some classical questions that companies asked or companies ask in their multiple choice online questions let's look at this question as a developer using this pay-per-use service you can send store and receive messages between software components which of the following is being referred here let's look at it right we have aws step functions amazon mq amazon simple queue service amazon simple notification service let's see the question again as a developer using this pay-per-use service so the service that we are looking for is a pay-per-view service you can send store and retrieve messages between two software components kind of like a queue there so what would be the right answer it would be amazon simple queue service now amazon simple queue service is the one that's used to decouple at the environment it breaks the tight coupling and then it introduces decoupling in that environment by providing a cue or by inserting a queue between two software components let's look at this other question if you would like to host a real-time audio and video conferencing application on aws it's an audio and video conferencing application on aws this service provides you with a secure and easy to use application what is this service let's look at the options they are amazon chime amazon workspace amazon mq amazon app stream but you might tend to look at amazon app stream because it's real time and a video conference but it's actually for a different purpose is actually amazon chime that lets you create chat and create a chat board and then collaborate with the security of the aws services so it lets you do the audio it lets you do the video conference all supported by aws security features it's actually amazon china let's look at this question as your company's aws solution architect you are in charge of designing thousands of individual jobs which are similar which of the following service best serves your requirement in earlier ec2 auto scaling aws snowball aws fargate aws badge let's read the question again as your company's aws solution architect you are in charge of designing thousands of individual jobs which are similar it looks like it's batch service let's look at the other options as well aw snowball is actually an storage transport service ec2 auto scaling is you know in introducing scalability and elasticity in the environment and aws fargate is container services aws batch is the one is being referred here that actually runs thousands of individual jobs which are similar aws batch it's the right answer but let's look at the other one you are a machine learning engineer and you're looking for a service that helps you build and train mission learning models in aws which among the following are we referring to so we have amazon sage maker and aws deep lens amazon comprehend aws device farm let's read the question again you are a machine learning engineer and you're looking for a service that helps you build and train machine learning models in aws which among the following are referred here the answer is sage maker it provides every developer and data scientist with the ability to build train and deploy machine learning models quickly that's what sagemaker does now for you to be familiar with you know the the products i would recommend you to you know simply go through the product description you know there's one page available on amazon that explains all the products are quick neat and simple now that really helps you to be very familiar with you know what the product is all about and what it is capable of you know is it a db service is it a machine learning service or is it a monitoring service is it a developer service stuff like that so get that information get that details before you attend an interview and that should really help to answer or face such questions with a great confidence so the answer is amazon sage maker because that's the one that provides developers and data scientists the ability to build a train and deploy machine learning models quickly as possible all right let's look at this one let's say that you are working for your company's id team and you are designated to adjust the capacity of the aws resource based on the incoming application and network traffic how do you do it so what's the service that's actually helps us to adjust the capacity of the aws resource based on the incoming application let's look at it amazon vpc amazon iam amazon inspector amazon elastic load balancing amazon vpc is a networking service amazon iam is an username password authentication amazon inspector is a service that actually does security audit in our environment and amazon elastic load balancer is a service that helps in scalability that's in one way you know indirectly that helps in increasing the availability of the application right and monitoring it monitoring you know how much requests are coming in through the elastic load balancer we can actually adjust the environment that's running behind it so the answer is going to be amazon elastic load balancer all right let's look at this question this cross-platform video game development engine that supports pc xbox playstation ios and android platforms allows developers to build and host their games on amazon's servers so we have amazon game lift amazon green grass amazon lumber yard amazon sumerian let's read the question again this cross-platform video game development engine that supports pc xbox playstation ios and android platforms allows developers to build and host their games on amazon servers the answer is amazon lumberyard this lumber yard is an free a gaming engine deeply integrated with aws and twitch with full source this lumbar yard provides a growing set of tools that helps you create and highest game quality applications and they connect to a lot of games and vast compute and storage in the cloud so it's that service they are referring to let's look at this question you are the project manager of your company's cloud architect team you are required to visualize understand and manage your aws cost and usage over time which of the following service will be the best fit for this we have aws budgets we have aws cost explorer we have amazon work mail we have amazon connect and the answer is going to be cost explorer now cost explorer is an option in the amazon console that helps you to visualize and understand and even manage the aws cost over time who's spending more who's spending less and what is the trend what is the projected cost for the coming month all these can be visualized in aws cost explorer let's look at this question you are a chief cloud architect at your company and how can you automatically monitor and adjust computer resources to ensure maximum performance and efficiency of all scalable resources so we have a cloud formation as a service we have aws aurora as a solution we have aws auto scaling and amazon api gateway let's read the question again you're the chief cloud architect at your company how can you automatically monitor and adjust computer resources how can you automatically monitor and adjust computer resources to ensure maximum performance and efficiency of all scalable resources this is an easy question to answer the answer is autoscaling right that's a basic service and solution architect course is it auto scaling is the service that helps us to easily adjust monitor and ensure the maximum performance and efficiency of all scalable resources it does that by automatically scaling the environment to handle the inputs let's look at this question as a database administrator you will use a service that is used to set up and manage databases such as mysql maya db and postgres sql which service are we referring to amazon aurora amazon elastic cache aws rds aws database migration service amazon arora is amazon's flavor of the rds service and elastic cache is is the caching service provided by amazon they are not full-fledged database and database migration service just like the name says it helps to migrate the database from on-premises to the cloud and from one database flavor to another database flavor amazon rds is the service is the console is the service is the umbrella service that helps us to set up manage databases like mysql money db and postgres sql it's amazon rds let's look at this last question a part of your marketing work requires you to push messages to onto google facebook windows and apple through apis or aws management console you will use the following service so the options are aws cloudtrail aws config amazon chime aws simple notification service it says a part of your marketing work requires you to push messages it's dealing with pushing messages to google facebook windows and apple through apis or aws management console you will use the following service it's simple notification service simple notification service is an message pushing a service but sqs is pulling similarly sns is pushing right here it talks about a pushing system that pushes messages to google facebook windows and apple through api and it's going to be a simple notification system or a simple notification service and with that we have come to the end of this video on aws full course of 2022 i hope you found it useful and entertaining please ask any questions you have concerning topics covered in this video in the comment box below our experts will assist you in addressing your problems thank you for watching stay safe and keep learning hi there if you like this video subscribe to the simply learn youtube channel and click here to watch similar videos to nerd up and get certified click here you

Channel: Simplilearn

Views: 477,285

Keywords: aws tutorial, aws full course, aws, aws tutorial for beginners, aws for beginners, aws training, aws course, aws cloud practitioner 2022, aws course for beginners, aws full tutorial, aws cloud, aws fundamentaals, amazon web services tutorial for beginners, aws cloud practitioner essentials, aws interview questions, aws ec2 tutorial, aws ec2 tutorial for beginners, aws lambda, aws lambda tutorial, aws s3, aws s3 tutorial for beginners, aws cloud computing, simplilearn

Id: ZB5ONbD_SMY

Length: 686min 42sec (41202 seconds)

Published: Tue Jan 11 2022