AWS New York Summit 2018 - AWS PrivateLink: Fundamentals (SRV211)

welcome today I'm talking about VPC private link my name is Gina Morris I'm an engineering manager on the V PC team the ec2 networking team if you don't know V PSD V PC is virtual private cloud I'm gonna cover some basics on V PC but if you haven't learnt a bit about V PC before I suggest that you go and have a look at our V PC fundamentals talk if you look on the AWS YouTube channel that'll get you up to speed with some more of the basics and allow you to fine tune of e PC but today it's private link so private link allows you to easily and securely use or offer or share services between your V PC and other V pcs you can also securely use AWS services from inside of your V PC without letting any of your traffic traverse the internet so what's our agenda what are we looking at today firstly I'm gonna tell you why why should you use private link then I'll go over as I said some V PC basics some of the fundamentals that you need to know to actually use private link now this is an intro talk it's a 200 level session so we're going to cover some of the basics and the easiest way to I think teach people about something is to do a bit of a walkthrough so we're going to do some walkthroughs looking at how you would share a service how you would use AWS services and then how you would actually consume or use a service that has been shared with you and then lastly I'll go over some example use cases so that you can actually see private link in action or kind of get a good feel for how it might be able to benefit you so before I show you how I'm going to show you why and there are multiple benefits of private link but I'm gonna focus on three of the main benefits to start with but I'm sure you can find more if you actually poke around and see how it can benefit your business so the first thing is that it's secure because your traffic does not traverse the internet it's really easy to use and it simplifies your network management especially at scale and lastly private link can help you to migrate your your systems from your on-premise location to the cloud or to operate in a hybrid model so private link allows you to create a private link between your own VPC and a service either an AWS service or a shared service running in some other VPC even if that other VPC is in a different account you get a private IP you get an elastic network interface and you can apply security groups to that interface this means you get to control exactly what traffic goes to that service endpoint and again your traffic does not traverse the internet so it's almost as if the service is running inside of your V PC with only private IP connectivity private link really allows you to simplify your network management and the way that you get this statement of its simplifying things for you is that without being a networking expert and I'm sure some of you are I'm sure some of you are really really capable you could do all of this already using some of the other building blocks that we've provided but with private link you're reducing your your you're reducing your opportunities to make small mistakes you're reducing the time you have to spend maintaining those you're not having to maintain special firewall rules you aren't having to set up path definitions route tables and if you're sharing a service with someone else either a customer another team within your department something like that you don't have to have them also deeply understand the core networking they can just use your service and get it right and stay secure in VPC without having to be a network expert and lastly but definitely not least this can help you accelerate your migration to the cloud and can help you operate partially in your own data center and partially in VPC so if you have your on-premise data center connected to your V PC using Direct Connect your private link endpoints your V PC endpoints in your V PC are going to work over that Direct Connect connection that means that you can stage your migration you don't have to do kind of a lift and shift and move everything over at once you don't have to go and connect to AWS services from your data center over the Internet you don't need to do any of that you can create private link endpoints VPC endpoints in your V PC and connect to them from inside your on-premise location you could also potentially keep all of your own data inside of your own data center but actually have your applications running in your V PC there are numerous ways that you can hook this up together to simplify things and allow you to do your migration or your hybrid model stage-by-stage so as I said there is an entire talk on V PC fundamentals it's actually called V PC fundamentals so it's easy to find you can look it up on YouTube and really I do suggest you watch that to help you get kind of the basics and to help you fine-tune your V PC for your own use case but today I'm going to focus on a few basics that are relevant for private link I'm going to go through these pretty quickly so hopefully it's enough for you guys to get the basics and hopefully it's not boring for those of you who've seen them before so we're going to cover subnetworks subnets and availability zones and how you can use those to deploy higher availability applications will look at routing traffic will look at once you can actually route traffic how do you stop getting the traffic that you don't actually want how can you be selective about that and lastly I'm going to touch on an element called an elastic network interface or an Eni we don't usually cover this it's a fundamental it's kind of a virtual networking device however it is very relevant to private link so subnets are as the name suggests sub networks of your VPC and they're how you use your V PC to deploy high availability applications and they're relevant for private link because when you are creating a private link endpoint when you are creating a private link endpoint service so when you're sharing a service using private link you want that service and you want that endpoint to be available in multiple availability zones for redundancy or potentially for lower latency so AWS has a global infrastructure we have 18 AWS regions and when you create a V PC you're selecting a region your V PC crosses an entire availability zone I mean an entire region availability zones are next so each region is divided up into two or more availability zones and each availability zone is made up of one or more data centers that have redundant networking redundant power they basically have completely separate failure characteristics so that if something happens in one you've got redundant setup in other availability zones and when you create a subnet your subnet is a sub Network of your V PC within an availability zone so we want to send packets and that kind of brings us to routing which is one of the core concepts of V PC now caveat routing isn't really directly related to private link however it is related to V PC endpoints and vaguely related to private link so I'm going to touch on it anyway so the way you do routing in V PC is route tables and route tables are a simple easy to read list of rules that says when traffic is destined for IPS in a specific range send that traffic to a specific gateway you can set that up by VPC but you can also override it on a subnet by subnet basis so here is a example route table and this is saying traffic that looks like it's going to the IP address range of my V PC should be routed local it should stay within my V PC that's pretty simple sometimes that's what you want you don't want your traffic going anywhere else any of your instances start trying to send traffic to the Internet you're gonna drop it but sometimes you do want your traffic to go to the Internet and so you create a route that says traffic that matches 0.0 0.0 0.0 it's a gateway that sends traffic to the Internet now it's not a single point of failure it's an abstraction over something highly available and sometimes this is what you want right sometimes you want to be able to send traffic to the internet but often you're in a situation where you don't want to do that now here's an example of I mentioned that you can when you create a route table by default it'll apply to your entire V PC but you can override that on a subnet by subnet basis so here's an example that shows how or why you might want to do that so we've got some yellow web servers and we've got some blue back-end servers and the yellow web servers take requests from the internet and in the course of handling those requests they turn around and they make a request to the blue back-end servers so we have two distinct groups two distinct groups of connectivity needed here the yellow web servers need to get requests from the internet so we're going to create a route that allows them to send traffic and receive traffic from the internet by routing to an Internet gateway the blue back-end servers they only ever need to get traffic from the yellow web servers they only need to know about that local route so we're not going to give them a route to the Internet now why do you care why not just have everything routable from the internet well how many of you have compliance or auditing requirements you can put up your hands if you like yeah you see it's pretty prevalent some of you might have those requirements and not even know it yet it's going to be a fun surprise so you might have compliance or auditing requirements where maybe you aren't able to or aren't allowed to send certain types of data over the Internet so you don't want your back-end servers which handle that data being routable from the internet others might just want to take a bit more of a belt-and-suspenders approach to security you just want to make doubly sure that nothing has ever happening that you don't know about and that's fine too and private link is really awesome in this situation because private link is not going to be sending your traffic over the internet so it doesn't need you to have a route to send traffic over the Internet you're able to preserve this not being routable behavior while still being able to use AWS services or services that are provided from another VPC so security is important we've just been talking about compliance auditing requirements often that's for security and so the way you do security in your VPC is by using security groups and security groups are a powerful yet simple tool that in a typical data center or traditional data center you would achieve using firewall rules in private link security groups are relevant because you can actually apply a security group to your private link endpoint and that means you can say hey these particular servers in my VPC are allowed to talk to this VPC endpoint and everything else is not allowed to talk to that service so if you think about it say you have you know connectivity to my logging service you could define a group of instances that are allowed to send traffic to my logging service and everything else won't work so let's look at that same example we had before our yellow web servers are blue back-end servers and because there are two different groups there are two sets of I guess different security considerations and all of the members of each group share a common purpose different rules apply we want to allow web traffic to our web servers and we only want to allow our web servers to talk to our back-end servers so the web service example is pretty simple we're going to say allow all HTTP traffic to instances that are in the web server group from anywhere right so they can get these web traffic requests we already spoke about them being routable from the internet that's fine how about this this is a little bit more interesting so this is the security group for our back-end servers and you can see the source isn't a set of a range of IP addresses it isn't the list of IPs of the servers that are in our web servers it's another security group I think about how flexible that is how powerful that is as a tool you're in ec2 the e stands for elastic you want to be elastic you want to be dynamic you want to be scaling up and scaling down and this makes it really easy because with security groups as you're launching instances changing the sizes of your instances as the IP is are changing you aren't having to maintain those rules manually they're just getting updated and you're just getting the access that you need between those two groups so you can really practice the principle of least privilege without ever having you know that sneaky little IP that was one thing becomes another thing right I'm sure anyone who's managed firewalls themselves in the past by IP is familiar with this problem okay elastic network interfaces also known as en is our virtual networking cards in V PC and fundamentally when you send a any kind of network traffic or when you receive any kind of network traffic on your instances on your load balancers that traffic is coming from and going to elastic network interfaces this is obviously between instances and between other resources in V PC so it's a virtual networking card it has a private ipv4 address it has potentially secondary ipv4 addresses it can have public IPs it has if you opt in to ipv6 you'll have ipv6 IPs you typically will create one or when you run an instance you'll already have an elastic network interface and in those cases you're actually managing the network interface and that means you have permission to do attachments and detach and move it around and so on delete it but certain resources like load balancers like private link endpoints when you create those resources we create elastic network interfaces in your account on your behalf and we manage those for you and what that allows us to do is have things that are running inside of your V PC acting as part of your infrastructure allowing you to apply your security groups for you to really have control but will take care of putting all the pieces together to make it work and the last relevant piece here is that you apply your security groups to elastic network interfaces so those are the main basics of V PC again there's a lot I didn't cover here I also went over that pretty quickly so if you want to learn more about V PC please go and watch the VPC fundamentals talk or have a look at the documentation available on the AWS site so let's look next at how to access how to securely access AWS services from within your V PC there are two ways of doing this and both ways are using VPC endpoints but there are two different types of VPC endpoints the first is gateway V PC endpoints and the second is interface V PC endpoints which is why I threw in the elastic network interface path there so let's have a look at gateway V PC endpoints before they work just like all of the other gateways in V PC and so we looked at that internet gateway earlier these are gonna work very very similarly and the key thing there is that you route traffic to them using a route table so let's look at your V PC without a V PC endpoint today I'm gonna use s3 for this example so let's say you have your application your application is running in ec2 and you have your data your data is stored in an s3 bucket now your data is kind of a part of your application write those two things need to work together or you need your data to be able to do your to deliver your application and so you need to access your data from your application now if you go and in your instances resolve the dns name for your s3 bucket you're gonna get a public IP which means you're gonna need an Internet gateway or some other way of going over the Internet to s3 to fetch your data but we've just spoken about this and we've said they're really good reasons why you may not want to or be able to go over the Internet why you can't have your traffic traversing the internet and that's where V PC endpoints come in so with Gateway V PC endpoints they support s3 and dynamodb and you send traffic to them as I said by routing so here's that same example we just looked at for s3 and fundamentally what's going to happen is you're going to create a V PC endpoint using the V PC endpoint console and you're going to or the api's for those of you who use the API is I'm sorry we're doing a lot of console today you can also use the api's and then you're going to create a in your route table and your route is going to say any traffic that looks like it's destined for s3 send that traffic to the VPC end point and it's just gonna work you don't have to change your application you don't have to update your code you're just going to start sending traffic to that VPC endpoint and if you're not using your internet gateway or your nat gateway for anything else that's it you can take it away you're done so the next type of VPC endpoints are interface VPC endpoints and these are VPC endpoints that are built using private link now interface VPC endpoints are supporting a bunch of services today I had account but since then we've launched a few more so we're just rolling out VPC endpoints interface VPC endpoints for new services all the time in fact I think in the last few weeks we launched support for sage maker cloud watch events a couple of others and what happens here is when you create a VPC endpoints so imagine this is the the same as that s3 example you're trying to use the ec2 API endpoint when you resolve the dns for that you're getting a public IP and so you have to go over the Internet gateway or something when you create a VPC endpoint an interface VPC endpoint what actually happens is we're going to create an elastic network interface in each of the subnets that you specified with the security groups that you specify and when you send traffic to those private IPs on those en is that traffic is going through your V PC endpoint and to your to the service that that endpoint is fronting so it's as if now the ec2 api's are just running inside of your V PC remember this is a highly available virtual device it's running on something highly available and it's not a single point of failure when you create a V PC endpoint an interface V PC endpoint for an AWS service you're gonna get DNS and you're going to get one DNS name that is regional so when you resolve that inside your V PC you're going to get each of the private IPs all of them together and you will also get zonal endpoints you'll get zonal DNS which means if you want to keep all of your traffic within a specific availability zone you can use those zonal dns names now you can see I had to put some ellipses in there because they didn't fit on the screen they're quite long they're made up of the V PC endpoint ID the service name and that's okay but if you are using AWS services we can make this even simpler for you if you have allowed AWS to manage DNS on your V PC this is a V PC setting will create split horizon DNS so we will set up DNS inside of your V PC that when you resolve the normal service endpoint it's going to resolve to the IPS the private IPS of your V PC endpoint and that means your software is just going to work it's just going to start using the V PC endpoint without you having to change anything so we we launched V PC endpoints for AWS services a while back but last year we gave you our customers the ability to share and use third-party services using V PC endpoints and AWS private link and this is across V pcs but also across accounts so I'm sure some of you are you know dealing with situations where you have a lot of V pcs I'm sure others are dealing with situations where maybe you have a bunch of different accounts that you've acquired either through acquisition maybe that's just your system designed to kind of keep things segregated and this allows you to actually start sharing services between those accounts and those VP sees so we're going to talk about how do you share a service as the service provider maybe your team or your company builds the logging service be monitoring service something like that how are you going to make that sure that that service shared across V PCs and across accounts once you've done that let's look at how are your users or your customers are actually going to use those shared services and then lastly let's have a look at as a customer of some third-party services how can you start using those third-party services over private link and these are specifically services that are available in the AWS marketplace you can of course use shared services in the normal kind of direct share model as well if you have third-party providers who prefer to do that so let's look at sharing across V pcs regardless of account and we're also going to touch on if you are a marketplace seller an AWS partner how can you make your software as a service offering available in the AWS marketplace over private link what do you get for that so let's look at sharing services today this is very simplified but let's say you have your service it's running in ec2 and you probably have some kind of load balancer in front of your service INRI in reality your systems are probably a little bit more complex than this but you know this is the basic idea and you're running in your V PC and you have a user or a customer who's running in some other V PC somewhere they've got their instances and those instances want to access your service they want to be able to talk to your load balancer now there are a few ways of doing this I'm going to focus on just doing this over the internet because we've kind of been using that along the talk so what you could do is you could give the VPC that the services in an Internet gateway and create a route to the Internet gateway so it can actually receive traffic from the internet give your load balancer a public IP all that kind of stuff and you're going to need to do the same on the user's VPC the consumer VPC now both of these V pcs are routable from the internet they're able to send or receive traffic from the Internet and at this point you would be able to use the service in the service V PC from the instances in the consumer V PC and a lot of the time this is fine right a lot of the time this is going to work just fine but as we've been discussing there are a lot of reasons why you might not want to or be able to do this and there are multiple options you know you can use peering you can use NAT gateways there are a variety of options but private link is the preferred way to share services directly between V pcs and that's because it's going to easily allow you to keep your data secure and it's going to allow you to really simplify the management especially when you're sharing your service with a lot of different users so there's only one real requirement compared to the setup we had here your load balancer in front of your service there's only one real requirement here and the requirement is that you have a network load balancer in front of your service now network load balancer is a TCP load balancer and you go to the load balancer console it's in the ec2 console and you just follow the wizard and create a network load balancer you select the availability zones we suggest you use multiple availability zones this allows you to have lower latency cuz you're in more different availability zones and it also improves your fault tolerance because of that redundancy and that's pretty much it once you've got your network load balancer you're going to go to the V PC endpoint service console and here you're going to follow the wizard and all you do is select one or more Network load balances so a key note here if you have a very large service or if you're wanting to kind of split traffic between multiple services that are you know a be testing or something you can have multiple load balances here all of your customers your users will be mapped to one of those network load balances so their traffic will go to one of the load balances that you have specified they won't be going between those load balances at this point the only other option that you have to look at here is this acceptance required and checking acceptance required means you're going to review requests from consumers requests from your users on a case-by-case basis if you don't check this every time someone is allowed to request to share or use your service it will automatically accept that request and voila that's your service it's created now your service gets created I'm gonna step through each of the tabs on the console view here and tell you what they're about but on this first details view the main thing to take note of is the service name because the service name is what you are going to use when you share your service with someone else this is what you are going to give them for them to be able to discover and find your service so in the network load balancers tab you can change or just review your different network load balancers in the whitelisted principles tab so you don't want to just randomly have requests from anyone so you have to regardless of whether you are requiring acceptance whitelist specific I am users or roles or AWS accounts to say when they whether they are allowed to make requests to use your service the endpoint connections in this example is blank because no one is yet sharing the service no one has yet set up or requested to share the service so we'll come back to this in a moment lastly notifications so this is super useful you can set up notifications so that you get notified every time someone for example tries to connect or makes a request to connect to your service and this is really useful if you want to automate some of that review and approval process that we spoke about when you're accepting when you're requiring acceptance and so on so if you are making your service available over marketplace if you are a partner and you want your service to be or your service maybe already is available in the AWS marketplace you can register to have your software fulfilled over AWS private link now there is a really large pool of customers who aren't who can't use third-party tools because of lack of trust in how that data is being shared not necessarily lack of trust in the third party but kind of what's between their VPC and that third party service now thousands of customers are using private link and if you're a third party service provider you want to tap into that pool of customers who can't currently use your service but who would be able to use your service if it was offered over private links so that's one really good reason to go and onboard in the marketplace another enhancement you get if you make your service available over AWS marketplace using private link is you get a vanity DNS name so by default when you create a service the base DNS name is made up of the service ID and when endpoints are created for your service they are made up of the VPC endpoint ID plus the service ID and so on if you're an AWS marketplace service you get a vanity DNS name so you get to simplify that and that makes it much easier for your customers to recognize your service and it also simplifies TLS integration or TLS termination on your service so let's look at how you actually use the service because that's pretty much all there is to sharing a service it really is pretty simple so how do you use the service as a customer so again you go to the create end point part of the console and it's really it's the same sort of thing as what happened when you created an endpoint for an AWS service except you're checking the second option you're going to enter that service name that was provided to you and you're gonna hit verify and verify is checking a couple of things it's checking is this valid service name have you made it up or is it actually a real service and it's also checking that your whitelisted to actually make requests to that service if both of those check out then you're going to go ahead and specify your VPC you're going to specify the subnets and the availability zones that you want to create that VPC endpoint in so you're going to select one subnet for each availability zone now remember you want to do this in multiple availability zones for redundancy and lower latency and then lastly you're going to select your security group and this security or these security groups will be applied to all of those elastic network interfaces that are created as that service endpoint in your V PC at this point you're going to see this pending acceptance and on the service provider side it's also going to say pending acceptance and it's going to show that until the service provider goes in and actually accepts that request once you accept the request on the service provider side it's going to start moving from pending into available and at that point the V PC endpoint is ready to use and it's going to be reflected on both sides you can see the DNS names exactly like we had when we were looking at AWS services when you send traffic to those DNS names resolves to the IPS the private IPS inside of your V PC and it's now as if that service is running inside of your VPC so if we look at it in pictures on the service consumer side you are sending traffic to this private IP and that's sending traffic to the endpoint which is going in turn to this cool shared service what's really happening behind the scenes is that traffic is going to that network load balancer on the service provider side and to their service and all of this is happening without you having to have any kind of internet access or anything like that to share that service across VPC it's secure and it's easy to use it's going to scale you also get DNS and if you want if you want to simplify this a bit you can use route 53 to make a cname of your choosing that will point to this V PC endpoint DNS name so we spoke about when we were talking about creating your service or sharing your service we spoke about vanity DNS names and other things that you get if you make your service available over the AWS marketplace so I'm going to show you just a couple of the extra steps if you want to as a user consume one of those services over marketplace or through marketplace so simply you start by browsing through the AWS marketplace and finding the service that you actually want to use you want to look for this little icon here which says fulfilled by private link that shows you that that service that software is available using private Lincoln next you're going to subscribe so far this is basically the same as anything else you've done to use third-party services over or through the marketplace so you're going to subscribe and then it gets familiar again you're going to go back to that create V PC endpoint console and you're just going to select the third option which is to use your marketplace service the only difference here is you're going to get a list of V PC endpoint services that you have already subscribed to using the marketplace now if you don't see what you're looking for in the marketplace offering private link you should reach out to them and ask them to unboard with private link because as you've just seen it's really simple to unboard so you know a little bit of a push in the right direction so that's basically the walk through of how to actually use private link it really is very very simple and it allows you to easily keep your data secure stop it traversing the internet and all in an easy and scalable way so let's have a look at three use cases of how you can actually use private link or how you can leverage it this is not an exhaustive list these are just a few examples that tie back to those benefits we spoke about so software as a service providers they often collect data and I know with the ones I've used before you install some kind of an agent on your on your instances say and that agent sends data or has an agent that sends data that reports back to the provider right they're gathering that data and then they're able to make some beautiful graphs for you and so on but there are customers who can't use those third-party tools because they can't send their data over the internet or they don't want to and they therefore have to choose between allowing internet access or not using this third-party provider with private link you don't have to make that choice your data is never going to go over the Internet so you can have no internet access and use that software as a service so we spoke a little bit about compliance so let's have a look at a specific example maybe so personally identifiable information or PII is specific information that cannot traverse the internet both PCI and HIPAA compliance have requirements that state this kind of data cannot go over the internet with private link you're able to confidentially share data because you know that there data is not going to traverse the internet so now you have a way of getting your PII data between your V PC and another V PC and lastly again not least migrating to a hybrid cloud model with on-premise applications you can now use private link to connect to AWS services or services running in other V pcs / Direct Connect this means that you can start to try out sort of how it feels to use various AWS services have part of your applications running in AWS without having to move everything over all at once having a staged migration really being able to tackle your migration one piece at a time is going to simplify your life so much and all of that while keeping traffic within your private network right so let's recap we've been through the steps of how to set everything up I spoke about some of the use cases we spoke about the benefits so I'm going to touch on the benefits again you have control of your data your data is secure it's not traversing the internet you can apply security group so you can really control exactly what can send data to those endpoints you can practice the principle of least privilege in another way in V PC private link is scalable it's easy to use it's easy to use both for you as a as a service provider or as a user now think about that if you're a networking expert and you know you know how to do this your customers may not know how to do this you also you don't want to waste time doing all the little things making sure everything's right when you can very very simply share a service and know that it is secure and you can accelerate and simplify your migration to the cloud or your migration to a hybrid cloud model in a lot of different ways by simplifying and staging your migration private link is the recommended way to share services between VP sees it's secure its scalable and it's really really easy to use now this was an introductory talk I went through pretty much all of the basics all of the different types of users that might be using it so if you are interested in private link there is a lot of information and some more kind of deep dive webinars and sessions online so you should definitely go and check that out if this is going to interest you thank you very much remember to submit sub session feedback thank you

Channel: Amazon Web Services

Views: 22,111

Keywords: AWS, Amazon Web Services, Cloud, cloud computing, AWS Cloud, SRV211, awsnysummit2018, nysummit2018, AWS Cloud Value Framework

Id: 20RxEzAXG9o

Length: 42min 49sec (2569 seconds)

Published: Mon Jul 23 2018