Microsoft Peering vs Private Peering and Private Link for Azure PaaS Access from On-premises

hey everyone in this video i wanted to address a question that's actually coming up more and more frequently and it's really what's the difference between using microsoft peering over express route to access azure pass services compared to private peering with a private link endpoint so in this video i'm just going to go over what both of those things actually are how they work and then hopefully you can understand the difference between them as always if this is useful a like subscribe comment and share would be appreciated okay so if you think about azure as we always talk about azure is made up of regions so i can think about hey i have a particular region and there are lots of them so there's lots of regions all throughout the world now these regions are connected remember to that great big microsoft backbone network so we always have kind of these redundant connections from each region and these regional network gateways that connect us to that backbone network and this microsoft backbone network is one of the biggest networks in the world now in addition to connecting to the azure regions that host things like azure and office 365 and being an xbox they also extend down to a large number of kind of edge locations you might call them kind of pops but again lots and lots of these edge locations have that microsoft network extended into those as well now at those edge locations they do a number of different things one of them is to actually go and connect to other carrier networks so i could imagine for example hey um that might be kind of a t they have a network so microsoft connects to that network of various kind of points and presence you might imagine that also hey look uh maybe verizon and and there's lots and lots of these carriers but again you can kind of think about microsoft connects to lots of these different carriers at those edge locations it tries to have a direct one-hop connection to all of the major carriers and that's what the internet is remember the internet is just hey at certain points all those different carriers come together you might hear them called these carrier neutral facilities fiber hotels that is what the internet is all the different carriers connect together over this controlled ip space so there's no overlap and then that that is what the internet is so that's great so microsoft connects at certain locations to all the different carriers to give azure and other services basically internet connectivity now we then have lots of services in azure so i could think about well i'm going to talk about storage a lot but this applies to really any of the past services via azure sql database or cosmos db or azure migrate the list goes on but essentially what you have is the idea that well i have a service so i'm going to say in my example i'm talking about azure storage now for those past services they are by default internet facing i address them via the internet and so the way this works is those services actually have a whole batch of ip prefixes and they vary by region so there's a batch of ip prefixes prefix is just a contiguous set of ip addresses so in south central storage uses a certain list of ip prefixes in east us to it a different set but essentially there's all these ip prefixes that make up a certain service and what happens is all of these different ip prefixes that make up the services remember they're internet facing so those address spaces are basically advertised out to all those different internet services that that's what they're doing now we can actually look at those so if i jump over for a second and what i'm going to do is quickly just show you so we'll often hear about service tags so service tags is a feature that lets me make certain subnets in a virtual network known to um that service endpoints to virtual networks service tags lets me control from network security groups saying how you can only access these services well it does that by looking at what are the ip addresses that make up a certain service so if i look at a service tag i can get a list of what are all the service tags and then i can say will show me the service tag values for storage in south central us and i could list out what are all of the different ip prefixes and as you can see there there's actually a lot of them there's a whole batch of these in fact if we count them out we can say there's 47 different ip prefixes that make up storage for south central us but fundamentally it's just a big batch of ip addresses and again those service tags were using network security groups to say hey my subnet or whether i apply this network security group can only talk to these services can only talk to these groups of ip addresses and you'll see there's the same for things like sql so i was showing storage but once again sql well it has its own list of ip prefixes just groups of ip addresses now sometimes you'll hear us talk about bgp communities so b2b community is saying we can kind of add on to the routing advertisements over bgp for a set of ip prefixes about what the service is doing and friendly name for it so we can then use that information that bgp information maybe to make routing decisions um for traffic modeling for example so we can see those same ip addresses i just showed you if i actually go and say hey i want to look at the bgp communities so if i get all of them for azure there's quite a lot i'll save it in a variable and then i look at the one for south central us well once again it's showing me all of those ip prefixes for south central us and notice it's exactly the same look at the first one 1365 107 32 well it's kind of those um same list of ip addresses if you look through you'll see they all match so it's the same ip addresses that we're using over here as well so bgp communities are going to kind of tie up with the service tax ips we have so fundamentally all we are doing here is there's a big batch of ip addresses that's it that we advertise out to the internet saying hey um to get to these ip addresses i'm going by the internet and that's all it is so there's different groups of ip addresses for different services in different regions nothing more than that so now if i'm thinking about that well if i'm on premises for example let's say i'm here and i'm using a particular carrier so i have an internet connection let's say up to a t so this is my internet path remember i'm seeing that advertisement coming over that path so if i'm now sitting there and hey i want to use that service i try and go to something in storage well i'm going to take that internet path to get to it that internet facing path and most likely i've got kind of a nat service at the edge of my network because this is all a private ip space doesn't work over the internet so i'm at it and so azure sees that request coming from whatever now ip address i have at the edge of my network if i actually try to look at a particular storage account we jump back over so i have a particular storage account in my environment so if we just try and look that up we can see it's oh okay 20.150 over here dot 38.4 that is the ip address of that particular storage account that's at 20.150.38.4 i would see it there we go so it fits within kind of that prefix that i could see in the bgp community and if i kept going up i would see that same there go 20 150.38 over there as well so it's really nothing more than that i'm just going to a certain ip address which has been advertised down to me okay so that's advertised to the internet and if i wanted to as i mentioned i could absolutely kind of lock that down because if i and this storage account a particular storage account which is what i was showing you there well i'm going to see the traffic coming from whatever nap service i have at my edge but i'm going via the internet and i've drawn this nice straight line but it really wouldn't be it's the internet so it's going kind of all jagged all over the place to get there eventually okay so where does express route come in a microsoft peering so remember what is express route express route is the idea that i as an organization now have a private connection to connect me to the microsoft backbone so at the same edge locations i could actually kind of drop lines it could be to the same one so maybe drop lines into my location now there's different types there's ethernet layer 2 there could be mpls there's different ways but essentially what it is now doing is creating a connection from my network to that microsoft backbone they're kind of doing a nice little cross connect for me in here i now have a connection to the microsoft backbone but that on its own it's nothing yes i've connected to the network but it doesn't actually give me anything so we have peering and there's two types of peering there's private peering which is used to connect a certain ip space to another ip space i.e up here in azure i would have something like a virtual network and that virtual network has a certain ip space and private peering would enable me to connect my ip space to the ip space of that virtual network because there would be kind of an express route gateway in there and it would connect them and then there's microsoft peering so microsoft peering enables me to connect to services that aren't in a v-net and office services could be but you have to get special exceptions for those they're generally better served by the internet and get this asymmetric routing that breaks things because packets go out one way and they come in another and it doesn't work but those services like the the storage the the sequel the cosmos i can do those so let's focus on that let's focus on microsoft peering first how does that work with this so in microsoft peering i create a microsoft peer configuration on my express route circuit now what actually happened i drew these two lines but absolutely before i do that i would have a set of nat gateways remember microsoft peering i'm connecting to services that talk on the public ip space so they don't understand a private ip space so i would have a set of net gateway ips so i'd kind of have this nat ip range which is all the ips of my gateways then i have a different set of ips for the bgp connections between them but what i get out of that fundamentally now is i get this microsoft peering so i now have this connection but on its own it's not really doing anything now as part of that microsoft peering i tell it which addresses i want to advertise to the microsoft network and that would be that now ip range they're the ones that are kind of internet that's right that are internet addressable i would never advertise these to the internet they only get advertised to the microsoft network if they were also advertised to the internet can cause a lot of problems because again you get into that asymmetric weird routing scenario so these are only advertised over that connection but now i have to tell the microsoft network hey i want you now to advertise certain services to my on-premises network and we do that with a route filter so if i jump over here what we do is we create a route filter and remember i talked about those bgp communities those groups of ip addresses well in a route filter i select them i can go through and select all of the services that i want to make available for example here i can see like azure storage hey look south central notice that community 12076 colon 52 and it's longer than that 5200 and an eight that's going to match what we saw when we dumped out that bgp community so essentially when i tell it to dump out that bgp community which we can see here what it's now saying is hey it's actually all of those prefixes all of those groups of ip addresses and i can go and add multiple ones i can basically go through so i create the route filter which is the list of all the prefixes i want to advertise over microsoft peering and then i attach it to that microsoft peering circuit so that configuration on the circuit so now i created the route filter remember which was the list of all of those bgp communities which was essentially all of those ip prefixes and now instead of only advertising out to the internet well now those ip addresses because they're part of my route filter they are now going to advertise down through my express route microsoft peering connection and it's going to be a shorter path than that internet path so now when my client wants to go to azure storage in south central us because i'm advertising a path via my peering which is shorter than that internet path instead of going out by my internet connection i'm going to go up through my express route circuit so that's all microsoft peering is it's simply saying hey those public ip addresses that normally are advertised out to the internet and still are i'm also advertising them down via my express route circuit but it's going to have a shorter asn path i either hops i have to go to to get there than that internet direction so hey i'm going to go by my express route and that's really it that's the whole point of what microsoft peering is doing i'm selecting groups of ip addresses that i want to advertise down now through my express route connection and the cool thing you can do at this point is remember you control the netting i have to do that netting on my side so i know what those ip addresses are they're the ones i'm advertising up through that microsoft peering configuration on the circuit so technically if i had a particular storage account for example so let's say i've got my kind of let's say storage account one i can absolutely modify kind of the network configuration firewall of that service and only allow in these ip addresses so yes it's public facing but i'm saying the only traffic allowed in is going to come from these ips that are the ones i'm nating the traffic through that goes by my express route circuit so i can absolutely do that if i wanted to okay so that's microsoft peering and again i'm saying storage pretty much you saw all the different services that i can do on that route filter just to give you an idea there's so so many different services available all of these different types of service you can see the backup cosmos sql lots of other things all available here that i can actually control so so lots of different things there and then some of them you notice i saw cosmos and sequel they're the ones that allow me to granularly control than the ones just hey i take the stuff in the region to push down as well so that's microsoft peering great advertising those public facing ips now for a private connection where does private peering uh where does private link come into this story so remember the point of private link is you can take this service and the whole point of private link is it takes an ip address from the virtual network that i'm projecting into and that private ip address then points to a specific instance of the service that that's really all private link does so it's private ip1 for example points to storage account one so it's an ip address from the ip space of that virtual network that now when i access it goes directly to that particular storage account now that's one half of it and if you remember when i did a lookup for my storage account you may have actually know it saying slightly weird so if i jump back over notice that hey i had kind of the regular name so it's always like storage account.blob.cor.windows.net for mine and yes it resolved to a public ip address because i'm on a machine that's just out on the internet so it responded to that public ip but it also had kind of this strange alias and it had one to private link dot blob dot core.windows.net because this storage account actually has a private endpoint enabled now what happens then is so the dns i also have a special record for that private link variant that actually was dissolved to the private ip address so if i jump back over and we actually look on this machine and on this machine i do exactly the same ns lookup command as i did on the other one it resolves to a very different ip address it resolves to its private endpoint ip because this machine is on a virtual network that actually has the private endpoint dns configured and resolvable so the other half of a private endpoint is yes i have an ip on the network great but now the dns name actually has an alias it becomes an alias to a private link variant of it and i have to be able to resolve that private link variant now there's different ways i can do that there's obviously services like azure dns so i can have azure private dns zones and it will offer to automatically kind of set up that that private link dot blob dot core etc and it will add the record for let's say storage account one and it will add well hey it resolves to that kind of private ip address whatever it is it would do that for me so that's azure dns and if i'm just using azure dns hey i i really don't have to do anything else that zone it creates would be linked to from my virtual network so now i could just do a lookup and it would resolve just fine or i could be using my own custom dns and i can do one of two things then if i'm using custom dns well i could think about either if i'm running my own dns service this is my dns box i could either add a zone private link that whole big zone dot core and i would add a record called storage account one and i would add in that ip address so i have to now maintain a list the ip won't change while that private endpoint exists but now i'm maintaining my own records i'm manually creating them or there's a special endpoint for this azure dns they're in in any azure virtual network there's this special kind of 168 dot 63. 129.16. ip is always the same on any virtual network in azure that is azure dns service so if i was hosting my own custom dns in azure so this dns server is in azure so that ip address is resolvable well instead of maintaining my own record my other option so or i could actually set up a conditional forwarder and i could have a conditional forwarder conditional forwarder says for this dns zone go and look up over there so i could have a conditional forwarder and i used a regular zone i would use the regular kind of blob dot core.windows not the private links obviously for blob core etc i want you to forward to the azure dns ip address again that works because the dns server is in a virtual network and can talk to 16863.129.16. so we have choices if we're using just regular azure dns i don't have to do anything it will offer to create this zone for me so it can resolve to that private endpoint ip address so i'm using custom dns in a virtual network well then either i have to maintain the private link i don't create the regular zone so i break accessing any other type of storage account but i create the private link variant of the zone privatelink.blog.court at windows.net and add a record for storage account one and the private ip or i can say hey for blog.cor.windows.net i want you to forward to the azure dns ip and that's what i'm kind of actually doing in my environment i've actually got a custom dns running okay so that that's great what does that have to do with on-premises nothing i'm sitting over here but remember at this point this is now just an ip address there's nothing special about it if i actually go and look at my configuration and so this is the storage account where i've enabled um my private link if i go to networking and it goes my private endpoint connections you can see i've created the private endpoint i've created one for my blob and one for files it's nothing more than just a network interface i can see it's ip configuration and then that's that ip address that you saw my record actually resolves to that 10.0.1.4 is what it looks like for something actually within that virtual network so it's just an ip address there's absolutely nothing special about this which means this is where we get into okay private peering so what does private peering do remember private peering is nothing more than mapping an ip space to another ip space so i could now say hey i'm doing private peering over this express route connection as well to that express route gateway so now i've got an ip path from on premises to this ip address which means i can access a storage account so if this is private peering now mapping an ip address space to a v-net and that v-net has a private endpoint in it i can get to that from on-premises now i'm drawing private peering with express route this would work with site-to-site vpn as well this is just an ip address there's nothing special about it but there's one caveat would this work as is um i can get to the ip address but i won't be able to get to the name remember i have to be able to get that private link special name which is not going to work right now so i'd have really two choices the same choices i actually had here um with a slight skew i probably have on-premises dns servers so i have a dns server here so one option would be i would create those private link zones so i'd create a private link dot blob dot core etcetera i would add a record for storage account one that would resolve to that kind of private ip address and now i'm fine because when i did the ns lookup for storage account1.blob.cor.windows.net it would be an alias now to the private link variant which i can map to the ip address and i now have a path to the ip address and it's just going to work but now i'm maintaining those records on prem on my dns which is not a huge deal maybe i don't want to do that so the other option is i do that conditional forwarding so once again we can do the or but i cannot conditionally forward to that 16863129.16 because i'm down here that ip address means nothing down here so if i want to do conditional forwarding what i would actually have to do is i would create a dns folder in my network and this would have an ip address let's say 10.0.1.20 and this would be configured to forward to the whole one six eight dot i'm never gonna remember this 63.129.16. so that would be a conditional that would be forwarding on this for kind of that the blob dot core etc so then that 10.0.1.20 dns folder and that works because it's in the v-net so we can get to the azure dns endpoint so my on-prem dns server which has an ip path to 10.0.1.20 it could have a conditional folder for blob and i use the blob i use the full and again i'm doing blob this could be sql or anything else that would now forward to dot 10.0.1 so hey if i'm trying to get to blob.cor.windows.net finish right now send that request to 10.0.1.20 so okay that dns server forwards it to this dns server 10.0 1.20 which you can get to because it has an ip path it says oh i'm forwarding that same request to 16863.129.16 which is azure dns that has the record it's a bit more convoluted um but it will get you there i mean it will have that path and it will work so they're really my options when i think about hey my my on-prem use of that the benefit of this is there is no public endpoint when i'm using the microsoft peering it still has public endpoints um again i can control it because i i know those net ips i'm using for microsoft peering i can still control that traffic but it has public endpoints and some people don't want that so the private endpoints i can essentially turn off all of those public ips don't nothing goes public to the storage account anymore i can only get to it from the private endpoint ip address but now i use the private peering or sites.vpn to get to it you just need to make sure the dns so either i add the zone for the private link or i conditionally forward for the the public as part of the zone which goes to the dns folder in the v-net which can then talk to the azure dns endpoint to talk to it now a downside of the private link is obviously there's costs associated with that if you think about i have to pay for the private endpoint which is i think it's like a penny an hour or something and i have to pay for the ingress and egress traffic so i do pay for the traffic as well also i'm paying for the express route gateway which now all my traffic is flowing through i can't use fast path to taste so fast path is where normally traffic going into the v-net has to go through the gateway equest never goes through the gateway ingress does fastpath says hey i'm going to bypass the gateway and just go directly to the resource fastpath does not work with private endpoints today so if i'm using this to get to my private endpoint i'm using up whatever capacity that gateway is i'm using up a chunk of that capacity um so i'm paying for the gateway i'm paying for the private endpoint and i'm paying for the ingress and egress for on that private endpoint communications with obviously microsoft peering i'm paying for the egress traffic now maybe your supplier charges you separately for microsoft peering over private peering that that can kind of vary by carrier um they're really the the big differences between them and hopefully that that makes sense what we're really doing here microsoft peering we're just advertising those ip address spaces over your express for our connection and so i'm taking that path because it's got a shorter path and i can lock it down because i know the net ips that we not the traffic to going via microsoft peering private peering and private link hey i'm connecting an ip space so then i can get to the private endpoint ip but i do have to deal with the dns either by adding a zone for that private link part and adding the record for the storage account or my sequel whatever else or i have to have a conditional forwarder to a dns folder in the v-net which can then talk to the azure dns endpoint um that's it i hope that made sense i hope that was useful and as always until next time take care you

Channel: John Savill's Technical Training

Views: 10,827

Keywords: azure, azure cloud, microsoft peering, private peering, expressroute, route filters, private link, private link dns, azure dns

Id: i3byrLaJiiM

Length: 33min 2sec (1982 seconds)

Published: Tue Feb 23 2021

Reddit Comments

[u/mplsdude612]

Solid video and explanations.