AWS re:Invent 2017: Creating Your Virtual Data Center: VPC Fundamentals and Connecti (NET201)

hi everybody welcome to reinvent welcome to creating your virtual data center in the cloud my name is Gina Morris I'm an engineering manager I work on an ec2 networking and I have been building features for you since 2010 on AWS so no matter what you do in AWS there are two things that are going to be in the background of everything you do the one is identity and access management so it's a good idea to go out and learn a little bit about that the second thing is VPC virtual private cloud it's how you do networking in AWS so what does it mean what is virtual private cloud well it's virtual it's your virtual network your virtual infrastructure is running in a virtual network peers for private it's private it's entirely under your control so you get to choose how it looks you get to control what can and can't talk to what and then see it's your cloud it's in the cloud it's your cloud it's your virtual private cloud so show of hands who here is a networking expert yeah there we go be brave there we go so you guys are going to see a lot of stuff that's familiar a lot of concepts that you're going to recognize but in pretty much every case you're going to find that it's more flexible you get better visibility and it requires significantly less maintenance than what you're used to everyone else who isn't a networking expert maybe you don't want to become a networking expert you want to use your time to get really deep in other areas you want to work on building applications to deliver value to your customers you don't want to worry about the network and I've got good news for you VPC is easy to use you're going to be able to get up get set and get going while and being secure pretty quickly in fact I think in about the first 2025 it's of this talk I'll give you everything you need to know to get started now there are a lot of you here this is an introductory level session 200 is the lowest this is 201 it's an intro session so if you are already an expert and VPC you're pretty familiar with how things work this may not be the session for you so letting you know ahead of time what to expect those of you who really want to learn about VPC you've come to the right place I got the second part which is using the clicker correct okay this is an ec2 instance it's a virtual server and it's running an AWS and the e is for elastic you want to scale this up you want to scale down you want more of them you want less of them so you're gonna have a lot of change you're gonna have this dynamic set of servers and it's running in our network it's running in the AWS network but because it's your infrastructure we're gonna let you run it in your network your virtual private cloud and you have control so you're gonna pick the IP ranges you're gonna pick specific IPS or specific things you're going to divide this network up into sub networks and we'll talk a little bit about how you use these sub networks for higher availability and you can also choose what parts of your network or your whole network what connectivity you give that network so what are we doing today what's the agenda so firstly we're gonna get you familiar with the basic anatomy of a V PC what are the pieces how do they do how do they work what are they for secondly I'll walk you through a basic example and the simplest example is an internet-connected V PC because that's gonna pretty much solve most of your initial use cases and that's going to be the first half of the the second half of the talk we're going to focus on different connectivity options other than just Internet access for everyone we'll also take a look at different AWS services and how they interact with your V PC so more than just ec2 so the simplest V PC I said is an internet-connected v PCL V PC with internet access and many of you in your AWS accounts will already have a V PC just like this it's called your default V PC and if it suits your needs and that's all you need you are welcome to use it that's totally fine but it still believes you to understand how V PC works so that you can make changes or you can you know do different things later and you can make sure that you're not gonna run into any issues later on so let's get started what do we what are we going to have to do to create this V PC first thing we're gonna choose an IP address range right we control it we pick it next thing we'll divide our network into sub networks or subnets and I'll talk about how you use these to build higher availability applications we're gonna add routing so that our network is reachable from the Internet and once it's reachable from the internet we're gonna talk about how do we make it only accessible to the traffic that we actually wanted to be accessible to so choosing an IP address range this is my network I get to pick it it's a private network so I want to pick a private range of IP addresses some of you are going to recognize this so I'll go through it pretty quickly we use this notation a lot in V PC and I'm going to be using it in this talk this is cider notation that's classless inter-domain routing and it's an ipv4 address followed by a slash followed by a number and the way it works is you write the ipv4 address out in binary so you get a 32-bit number and then the slash sixteen means you hold the high 16-bit steady and the low 16-bit can vary so this gives us the IP range 1 7 2 dot 3 1 dot something dot something all of those IPs are in this IP range now this was an ipv4 example I'm gonna use a lot of ipv4 examples in this session VPC does support ipv6 and there is a session later today that focuses specifically on ipv6 so this is the range I'm choosing 1:7 2.3 1 dot something dot something so the question is why it's the first thing you want to think about is following convention RFC 1918 is an RFC for private networks so to make it easier when other people have a look at my IP addresses they understand what they are I'm gonna follow a convention the second thing we've done is pick a slash 16 it's big it gives us over 65,000 IP addresses to use now you may not really need that much space on day one that's totally fine it doesn't cost you anything and it gives you so much room to grow to move things around so you might as well just go with a slash 16 now one other thing to bear in mind is do you have other networks that you may want to connect your VPC to do you have an unprecedented do you have other VP C's either in York or your account or another account take a look at what those IP ranges are and try and pick a range that doesn't overlap because that'll prevent conflicts later on and if you're not sure you can start with a smaller IP range because you can come back later and add additional non contiguous IP ranges to your V PC right so we've got IPs let's talk about subnets subnetworks they're basically a sub range of that IP address space that we made for the V PC and they are how you use V PC to deploy high availability applications so AWS has 16 regions and AV PC is within a region right so in this example I'm going to use our Dublin region EU West one and an availability zone is - I mean a region is divided into availability zones so we've got a u.s. 1a 1b and 1c an availability zone for those who don't know is kind of like a data center it's got redundant power redundant networking it's designed to have completely separate failure characteristics from any other availability zone so you can see how you would want to build your applications across multiple availability zones so when you create a subnet a subnet is within an availability zone and you need subnets because when you run an instance when you create certain AWS resources you're going to be specifying the subnet that you want those resources to be in so having subnets is how you get your resources into a specific availability zone so we pick ranges just like we did for the VP c-17 2.31 dot 0 dot something 172 3 1.1 dot something same thing for us 1c so to recap subnets V PC sizing / 16 a really nice big V PC for your subnets we recommend a / 20 for those of you who are good at math we'll be saying a has an error on the slide it should be 256 it actually should be 251 because we reserved 5 IPS and then we also recommend that you create a subnet in each and every availability zone so that you can launch instances into every availability zone now you can add and remove subnets you can have more than one don't worry too much about it remember we've got the slash 16 so we have a little light subspace right so it kind of brings us to the point of a network which is sending traffic routing so how do we send traffic to the internet so routing in your V PC is done with wrap tables and route tables are a simple easy to read list of rules that say traffic destined for this destination should be sent to this gateway when you create a V PC you get a default route table and that applies to everything inside of your V PC but you can override routing on a subnet by subnet basis and we'll go into that in a bit more detail shortly so this is a look at the console route tables here is the rule so the default the rule that's created by default in every every route table says traffic destined for the IP address range of this V PC should be routed local it should stay within the V PC and if it's routed in if it's if you're trying to send traffic anywhere else there's no rule so it's gonna go nowhere the traffic will get dropped and sometimes that's what you want but not in this case in this case we actually want to send traffic to the internet so to send traffic to the internet you create an Internet gateway and you attach this to your V PC and it's pretty simple it's a gateway that sends traffic to the Internet so we go back to our route table and we add a route and the route says everything that matches 0.0 0.0 0.0 tation for match everything it's a default route everything that matches that should be sent to the Internet gateway to the Internet now the order doesn't matter of the rules it's doing most specific route so you don't need to worry about that the other thing to think about here is I know it says internet gateway it's not a single point of failure this is an abstraction backed by something highly available so now we're sending traffic to the Internet but we don't really want to get traffic from just anywhere right so that brings us to network security and EWS VPC has some pretty powerful yet simple tools that allow you to do network security and and this is the kind of thing that you would in a traditional data center do with firewalls so here's an example we've got some web servers and our web servers accept requests from the internet and in the course of handling one of these requests they turn around and they make a request to some back-end servers right so we've got our yellow web servers and our blue back-end servers and they share each group of instances shares a common use a common purpose so different rules apply to them they're in different groups different security groups so we can create these security groups put the appropriate instances in them and we can say allow web traffic from everywhere to my web servers but for my backends only allow traffic from my web servers from instances in my web service group this is how this looks in the ec2 console you create a security group and your security group has a rule that says allow all HTTP traffic right so you control the protocol you control the port and you control the source from everywhere you'll also see here I've got an IP six example just to show you that it works exactly the same it's pretty simple second thing I wanted to call out because this is new since last year is that you can add a rule on each you can add a description on each and every rule which makes it so much simpler when you're maintaining these groups when multiple people are looking at them when you want to remember why did I create this what you know what does it do so let's look at the back ends these are a little bit more interesting because they're not just allowing everything so in this case you'll notice that the source is not a range of ips it's another security group this is so flexible you're basically saying allow traffic on port 2 3 4 5 from any instance in this other security group in the web servers security good that is a zero maintenance way of keeping the the principle of least privilege of keeping all of these things up-to-date without you having to do anything you can add and remove IPs right because in a traditional data center you would have to go in and change the config time you created a new server you remove a server you got to go in change the configure the things are only talking to the right thing carrot just works it just works automatically and an ec2 we want you to be elastic so this is super super powerful so security groups follow the principle of least privilege only allow things to talk to things that they should be able to talk to and don't allow anything else both of those examples were for ingress rules incoming traffic you can create rules for outgoing traffic as well but by default they just allow all outgoing traffic right so remember at the beginning I said that I would give you everything you need to create a V PC get productive and secure in hey see - this is the point basically this is everything you really really need to know this is the basics and you can now be productive and secure very important so we got this Internet connected V PC so what other options are there for connectivity so the first one is restricting internet access maybe you don't want your entire V PC to be accessible from the internet the second is if you have an architecture that has multiple V pcs either in your own account or maybe you have multiple accounts each with a V PC and you want to pair those together we'll talk about V PC peering which allows you to achieve that and then last but definitely not least if you are migrating to AWS or you want to run a hybrid infrastructure there are a couple of ways of connecting your V PC to your on-premise data center so restricting access we spoke about how the default route table applied to the entire V PC you can also route by subnet so let's have a look at our webserver example this doesn't replace security groups you still want to use the security groups you created I'm not putting them in the diagram just kind of simplify things but remember we have our yellow web servers they're getting traffic from the internet when our blue back ends they're only getting traffic from our web servers so you can put each group into their own subnet because the subnet is the unit that we're doing routing on and we can say the web servers have a route table that has a route to an Internet gateway but the backend servers don't they can't route to the Internet and one reason you might want to do this is if you have any kind of compliance or auditing requirements that do to show that your servers are not routable from the internet this is one way the other reason people do this is just to take a real belt-and-suspenders approach to security having security groups and just not allowing anything through that shouldn't come through in our documentation and in the rest of this presentation I'm going to refer to a subnet that has a route to the internet or to an Internet gateway as a public subnet I remember this because you need a public IP to talk to the Internet sort of public subnet and then one that has no route to the internet gateway is a private subnet so let's look at another example here so if you have a private subnet you've got some very important stuff it's running in a private subnet and you need to maybe reach out from those instances to a young repo to get updated software you can't do that you have no route to the Internet and what you usually do is in this case is your dues network address translation so you could run an instance in the public subnet let's click on one you could run an instance in the public subnet and basically proxy traffic through that network address translation it takes the private IP that you were coming from write your source is a private IP and it rewrites it to a public IP sends it to the Internet and when the response comes sends it back that's fine you can do that we even provide amis so that you can go and and launch a NAT instance pretty easily but it's much easier if you use a NAT and Gateway and that gateway is a gateway that you send traffic to when you want that traffic to be NAT it to have network address translation so you create this an ec2 in the console using the api's you specify an elastic IP address that's a public IP that you control and then all traffic coming from your private subnet that you wrote to that that gateway is going to appear on the internet as if it were coming from that public IP and it works exactly like the Internet gateway in terms of routing in your route table you add a rule that says everything destined for the internet everything destined for everywhere send to the nat gateway and it just works and again this is not a single point of failure this is something highly available and it's a virtual device for your purposes so this last example on this topic is using both security groups and subnets we've got our private subnet we've got a bunch of instances there we've got a public subnet nothing really important runs there we really really want to isolate stuff but we want to SSH in we want to be able to SSH in and troubleshoot and see what's going on but we can't get there because we have no route to the internet so what we can do is we can create an instance that's an SSH Bastion that's kind of like a proxy for your SSH connections and this SSH Bastion is going to be in a security group right so I can SSH to that Bastion because it's got access to the internet it's got a route to the internet but I'm going to put it in a security group and say only allow SSH connections right 4:22 from a known IP range I don't want just anyone being able to SSH to my bastion instance I want to say when people are in my data center they can SSH to my Bastion instance and from the bastion you can SSH into your into your remaining fleet right anything coming from the bastion is allowed anything coming from your corporate IP or your known IP range is allowed so the subnet would have just been a big yes no the security groups allow you some finer grained control on on port on source on protocol right who here is thinking of an architecture that maybe has more than one VPC in it or more than one accountant it's pretty common yeah I see a lot of hands there we go so when you have multiple VP sees and you want to share either some resources some services across those you have one option which is you can connect both of those VP C's to the Internet and you can send traffic out of one across the internet into the other but as we've just discussed you have many reasons why you don't want to just be opening things up to the Internet so that's where VPC peering comes in VPC peering gives you private IP connectivity between two non-overlapping VP sees right this is where that IP range selection is important this kind of allows you to build as in the picture a hub-and-spoke architecture you've got one connecting to another one so VPC peering non-overlapping IP ranges gives you full private IP connectivity and one of the things that's really cool is your tools of least privilege your security groups just work here they work exactly the same so you can continue to be secure even though you've gone and joined these two networks now you don't want an appearing connection just to be created because one side said so so the process to actually create appearing is the our English side would in this case the orange side would initiate the request say I want to appear with the blue DBC the blue VPC would accept that peering request if they also want that connection to be opened at that point you have a peering connection but no traffic is going to be sent until you do yes add a route so the peering connection PC it behaves just like a gateway you say traffic that looks like it's destined for that other VPC over there send that to the peering connection and it's just going to work now those of you looking closely at the slide will see that there's a a gateway or a target type that we haven't discussed yet the vgw that's a virtual private gateway and that's what we're going to talk about next so as I said if you're migrating to AWS or if you have a hybrid deployment it's very very useful to be able to connect with privately connect your unpromoted a center with your V PC I'm gonna take a brief segue here this is kind of the genesis in some ways of V PC because back when ec2 launched initially there was no V PC so when you launched an instance you ran your instance in the ec2 Network and it was a 10.8 range and you got given the choice between Internet connectivity and Internet connectivity so you got Internet connectivity and you know you had security group so you could be secure like it was pretty good and this this is the ec2 classic network it still exists today but we started having customers who wanted to connect their own data centers to their resources in ec2 but they were running in a tender type e range and we were running in a tender P range and those two things did not mix and so that's kind of where V PC was born we realized that customers really want to be able to control the network on both sides of that connection so V PC is exactly that it's allowing you control of your network in ec2 so there are two technologies for doing connections between your on-premise network and your V PC the first is VPN virtual private network and the second is AWS Direct Connect I'm gonna explain these at a very high level when you might want to use each but they're massive topics all on their own and so if this is something that is useful to you or interesting to you you should definitely learn much more about them and go to some of the sessions they are awesome and super useful okay so high level overview of VPN in your own data center you will configure one of your own devices say a router and you will use we've got some step-by-step instructions in our documentation we support a wide variety of vendors each multiple devices for each and so you'll set up this router to be your customer gateway once you've set that up you give us the statically routable IP address then on the V PC side you create your virtual private gateway your vgw and if you've done this correctly and everything is lining up you're gonna get to IPSec tunnels so that you can send encrypted traffic from your on-premise data center to your V PC of the Internet now we recommend that you go and configure and set both of these tunnels up because they terminate in two separate availability zones allowing you to take advantage of AWS as global infrastructure the last thing to do here is add a route to your route table that says traffic that looks like it's destined for the IP range of my on-premise network send it to my virtual private gateway and that's all there is to it works pretty similarly from the V PC side for Direct Connect however instead of it being encrypted traffic over the Internet as it is with VPN Direct Connect is literally plugging a cable from your equipment into our equipment in one of our many colocation centers around the world which one you use depends on your data transfer needs so if you have really high data transfer needs Direct Connect is going to be the best bet and we do have customers who use both sides is for highest availability right so that's V PC that's a bunch of different connectivity options for getting between your V pcs getting to the Internet now I want to talk a bit about how V PC relates to the rest of AWS now hurts my feelings a little bit but I suspect you aren't here reinvent and you aren't interested in AWS purely because of networking purely because of EPC I suspect you're really here because of the wide variety of services that we offer and and all of the exciting new stuff that's coming out so I'm going to talk a little bit about how V PC and all of those other services interact we're going to talk about DNS you can do some cool stuff in your V PC with DNS we're going to look at some services that actually run inside of your V PC and I'll talk about a pretty cool set up we have for getting private access to AWS endpoints from within your V PC and then lastly I'll show you how to get visibility into all the stuff we've discussed and really understand what is going on so one very basic service of a network is DNS which allows you to do domain name resolution and in V PC you can do split horizon DNS so this is a look at the V PC console and there are two DNS options here and you want to say yes to both of them because it gives you some some cool stuff and we'll touch on some of those things in a moment the first is dns resolution do you want Amazon to take care of dns for you you can run your own DNS server but most of you probably don't want to or need to the second one is DNS host names DNS host names means that every time you launch an ec2 instance you'll get a host name and you get all of this without having to operate any machinery yourself so this is a look at Amazon route 53 this is Amazon's managed DNS service and most of the time people use this for public facing DNS but you can also use something called a private hosted zone which allows you to take over a zone inside of your V PC so for example I have taken over demo hosted zone org I don't own that but in my V PC I control what DNS does I can make demo hosted zone dot-org do whatever I want so in this example I've set up example dot demo hosted zone dot org to 0.217 2.3 one 0.99 that's a private IP in my V PC so anyone any server inside my V PC tries to access that DNS name it's gonna behave differently than it would if we were outside in the internet that's pretty cool so lots and lots and lots of AWS services are running infrastructure on your behalf so it's your infrastructure and because it's yours we're gonna give you the choice to run it inside your own network in your V PC now there are quite a lot of services that quite a lot of examples of this so I'm not going to go through every single service that does this but I will go over to patterns that you're going to see repeated over and over again and specifically these patterns exist to take advantage of the high availability infrastructure and multiple availability zones so first example is with Amazon RDS database when you create an Amazon RDS database you're given the option to run the database in your V PC and the first thing you're asked about is security groups your tools of least privilege and these work just the same as they do with instances you specify what can and can't talk to what and they just work the second thing you're going to be asked about is subnets and when we ask you about subnets what we're really asking you about is availability zones and so it's a good idea to specify more than one availability zone so that you can be distributed over multiple availability zones for higher availability so with the IDS example if you specified multiple subnets in different availability zones you'd get a master database in one zone in one subnet and you'd get a failover candidate standby in a second zone and this I call the act of standby pattern right specified multiple availability zones one active resource and one standby resource ready to come along if anything else anything happens to the first the other thing to call out share is you get a DNS name that's always going to point to the active database so the second example is application load balancer this is a layer 7 or an HTTP load balancer run by l.b elastic load balancing same thing security groups they work the same and again you're asked about subnets but what you're really being asked about is availability zones except in this case when you specify multiple availability zones that your application load balancer is actually going to create nodes in multiple availability zones so that it's constantly load balancing across multiple availability zones for higher availability and you're going to see these patterns or eat over and over again for various different services so VPC endpoints V PC endpoints give a direct private connection to AWS services from within your V PC it allows you to continue to practice the principle of least privilege also in some cases you get some pretty interesting tools for access management so I'll go into that in a little bit more detail as well so this is how the world works today I'm gonna use s3 as the example here you have your application it's running in V PC happy days you've got your data your data is in your s3 bucket and your data is kind of a part of your application now when you resolve the the s3 buckets dns name you're going to get a public IP address so if you want to access your data in s3 you're gonna have to create a way of going over the Internet either an Internet gateway or maybe in that gateway something like that and when you make the request you're going over the Internet to s3 to get your data and bring it back to your application and VPC endpoints exists to solve this so there are two types of VPC endpoint the first type is a gateway VPC endpoint gateway VPC endpoints are supported for Amazon s3 and DynamoDB and the way they work is you create a VPC endpoint and you put a route in your route table they behave just like all the other gateways you put a route in your route table that says traffic that is destined for this service s3 in this example will be sent to this VPC endpoint again not a single point of failure something highly available backed by AWS and that's really all there is to it it just works you don't have to change your code you're not pointing at something specific it just works another thing that you get with a gateway V PC endpoint is the ability to add Identity and Access Management policies on the VP's V PC endpoint and this applies directly to the endpoint so you can say exactly what the V PC can do and cannot do in each service that it's talking to with s3 you also have the ability to add a policy on the s3 bucket which means you can actually lock down that back and say this bucket is only accessible from traffic coming through that V PC endpoint it's really really cool if you want to lock things down so the second type of V PC endpoint this is pretty recent this a couple of weeks ago we launched this is an interface vp v pc endpoint and you'll also hear this spoken about as AWS private link for AWS services so this works very differently it is a V PC endpoint so you create a V PC endpoint and when you create a V PC endpoint you specify subnets and it creates an elastic network interface in each of the subnets that you specified now that elastic network interface has a private IP and when you send traffic to that private IP it goes to the V PC endpoint and directly to the service so you have private IP connectivity to AWS services from within your V PC now because this isn't a gateway you don't use a route table to send traffic to it you get given some DNS names you get one DNS name that'll have all of the IPs and then you also get given a zonal DNS name for each specific IP so the if you do want to keep traffic within a zone you can but this is you know these are IP I'm DNS names and they're lots of them and you would have to change your code so you can also choose to let us manage a DNS name in your V PC that looks exactly like the DNS name of the service outside of your V PC so when you resolve that DNS inside your V PC you'll go to your IPS your V PC endpoint and if you were outside you'd go to the public endpoint so again you don't have to change your code you don't have to do anything it's just going to work so there's a lot of stuff going on in your V PC you've got security groups maybe stuff isn't working as you expect how do you get visibility into your V PC into your V PC so you do this with V PC flow logs V PC flow logs get aggregated metadata about accepted and rejected traffic and allow you to easily go and look at that they really help you tell what your security groups are doing you don't have to manage any infrastructure for this either this just works and it's pretty simple to set up to you go to the VPN C console to the flow logs tab and there's a button you fill in some stuff and then you'll start getting your V PC flow logs pushed to Amazon CloudWatch logs and that looks like this so let me draw your attention to this entry over here this is a reject so it's a bit more interesting than most of what's going on on the rest of that page so we can see that this was some rejected traffic and we can see that it was UDP port 53 which some of you may know is DNS so someone was scanning my instance trying to see if I was running a DNS server and my security groups don't allow this kind of traffic so this got rejected it dropped but you know who is doing that why were they scanning it well we can do a reverse look-up on the IP address and we can see that internet police dot Columbia were scanning for DNS servers and they happen to hit one of my instances cool I guess I can live with that all of the except traffic anyone who's wondering we expect that we've got a load balancer running where load balancing traffic on port 8080 we want to see some accepted traffic we want to know that our traffic is getting through successfully cool right so to wrap up what have we talked about what have we learned today so we learnt about your network and your VPC is your network and we spoke a bit about your resources and AWS your instances and your other types of infrastructure we spoke about how to do networking in ec2 and how to use subnets for higher availability applications we spoke about security very important how to control who can talk to whom in a low maintenance way and following the principle of least privilege we spoke about running AWS services inside of your V PC and we spoke about getting visibility with AWS flirt with V PC flow logs we also discussed a bunch of connectivity options connectivity to the Internet with Internet gateways to your on-premise networks with VPN and direct connect to other V pcs using V PC peering and then to various AWS services using two different types of V PC endpoints all of this gives you visibility gives you a flexible and zero maintenance network and it's entirely under your control now there are a bunch of other sessions I mentioned a couple couple of them during this talk but many of them will give you a deeper look and some cases actually explain to you how some of the stuff works when you look a little bit deeper thank you very much at this point if I've done my job if I've done my job this this year I won't see you back here next year but you will come back to reinvent cool we'll be upfront for questions if anyone has thanked you

Channel: Amazon Web Services

Views: 42,505

Rating: 4.9608612 out of 5

Keywords: AWS re:Invent 2017, Amazon, Networking, NET201, AI, Connect, VPC, AWS Direct Connect, Security

Id: Tff1mekxOJ4

Length: 46min 32sec (2792 seconds)

Published: Wed Nov 29 2017