OpenStack Routed Provider Networks- What, why and how.

glad to be here Johnson I presented to a couple of backups past year so it's a little bit about Romana the project I've been involved with which is a network and security automation solution not going to talk a lot about that it's going to show up toward the end but the main top of the main topic of my presentation night is going to be open fact routed provider networks or layer 3 and routers and I kind of stumbled upon them I went to an old neck summit back in Vancouver I think it was and I learned a little bit of thing I kind of completely forgot about it and it sort of just bubbled up on my radar and got me very interested in I you know got very deep into all the different moving parts of routers provider networks because it actually solves a very interesting problem and and that's really one of the lessons I've spent a lot of time on talk about tonight the problems that solves how it does that and then at the gradient we'll talk a little about for mana but that bill pick it up in talk about how storage sort of relates to all things as well on so quick agenda we'll talk about the difference between provider networks and what you're probably most familiar with that open sandwich on project networks as tenant networks so the very big difference between those two and you have to understand why they're different in how they compare with one another and we'll talk about the existing provider networks that are typically VLANs or layer 2 provider networks and we'll talk about two different use cases the primary use case you look at the blueprints of the open stack what's happening once that you'll see the blueprint of the original open stack start the original run provider network use case talk about that and that's the primary use case that drove the development of the whole feature is that a new time um but on what I'm also going to talk about is a different use case that is the one that actually grew me into this whole topic which is one that I came across through some interactions that we have that I've had with a large operator running um Romana in their kubernetes deployed they also run OpenStack nevertheless I'll talk about that the second use case because I think it's quite interesting and relevant so those are things we'll go into and feel free to interrupt me we've got my half an hour to go through this should have never presented the material before so I think it will run about a half an hour but if you can go off-road and dive into any technical details if you exposure so let's start with a very basic comparison between provider networks and tenant networks and just so I show a hands who knows what a provider network is okay less than a third but 25% so that doesn't surprise me because all the provider networks are quite important and they sort of gets a sort of sidetrack because most people generally talk about tenant networks which are the user configured isolated layer 2 networks that are created through horizon or physics man line and as a user open stack those the networks that I provision and manage on my own so but that's what provider network provider network is literally a network that the open-back operator or provider that operator provides to the users and that's typically an existing data center feeling and that VLAN couldn't have a very specific purpose it could be the VLAN through which external Internet resources are available or it could be a VLAN that has assets that need to be accessed directly through OpenStack virtual machines databases you know very simple example you need to learn to adjacency for example leave those into the virtual machine you would build a provider network and expose that into the hosts and then the virtual machines would connect directly to the VLAN through a bridge on the host out to the data center network so that's what a provider network is now up to some very important use cases a very simple networking solution where all VMs in OpenStack hosts just you know grab an IP address on a datacenter viewing that was perfectly fine and I think probably one of the original simple days of OpenStack probably just had that for Neutron go back to like the Diablo games it probably stitches little adders there but important point is that this provider network is shared across all hosts across all projects so the IP address range is shared among all of those so in any protein that gets popped up it's going to catch an IP address out of a block that is available to all hosts across all projects so that's an important characteristic at the private provider networks and because of that I would just be clear you can't have overlapping IP addresses the way you normally do in tenant networks that are isolated VLAN that'll be a fight over ways so sometimes that's a good thing because you want any virtual machine to all virtual machines to access the public internet for example or access the Oracle database or an access of Active Directory or something so there's a lot some really good reasons why you might want that more than is why you would not like um so that's one of the limitations um but also important to know that if everybody's sharing this network they open to each other all these phone chains from all projects can access that same network so they can see each other people in security let's not go into that right now with the that's one aspect of providing networks and again these provider network used all the other data center infrastructure routers gateways firewalls and so forth now compare that with a tenant network which is the thing you're probably more familiar which is the network that a tenant builds they build a subnet they add the specify IP address range they launch machines on the network that give an IP address from the DHCP be that you've all done up a good stuff that a user is familiar with them at their private network in order to get off that network they've got to set any time router that's got to get routed out an external network and all of those things from a project intended perspective so again these are there two different use cases important for very different reasons but country just to reiterate the ten network is isolated virtual network created by the user it's usually deployed as an overlay Network using the X line technology you can use a portal again of overlapping RP addresses because every tenant can just specify whatever IP address they want you may have an that to get off of that but that's again the floating IPS would provide that access and again it uses Neutron routers or so so these are two very different networking models and we're going to talk about provider networks now so I mentioned this already which is also drill down a little bit to find out what's actually happening in the current layer to VLAN provider networks and what you see here in the diagram right out of the documentation what you see here is a is the Neutron network node and over here you have a compute node and here's a virtual machine that gets launched and the point the important point I want to make here is that what you see between the virtual gene that gets launched instance that get launched the connectivity between its network interface and the actual data center VLAN is a bridged Network so there's a provider bridge that goes out to interface on the host it gets connected to the actual physical infrastructure and this each zero on the instance is actually bridged to the data center viewing that means it's actually a layer 2 broadcast domain this link and art to get an IP address they can arc to learn the IP addresses with all these other IP data set of resources and standard vanilla deal and networking you'll see over here is a DHCP on daemon running and this is where the IP addresses are issued for this provider distance when Neutron launches is when Nova launches this instance it's going to fetch an IP address using DHCP and it can only do that on a bridge network so what you see here is another bridge to access the DHCP daemon to give its IP address so that's again meat-and-potatoes VLAN layer to network and that's how provider networks work today so the problem is not surprising the typical deployments for provider networks look something like this where you have the VLANs like I said there's VLAN 100 un 200 and you you uh connect those or plumb those VLANs so whatever Toprak switches you want and then down the rack and connect to whatever hosts need access to those feelings that all works just fine up until these VLANs get too big when you literally have hundreds of endpoints our paper addresses you know our timeouts the broadcast traffic and sometimes overwhelm the the VLANs and performance starts to deteriorate a VLAN is a notorious of fault domain and if something goes wrong all these endpoints are going to suffer so as the broadcast domains grow you're taking on greater and greater risk as these deployments scale and just generally understood that large VLANs are not the safest most robust deployment pattern but anyway they're quite common and and popular and and this is the way they would work so as I said that's how you want you is a but in reality you might have some crazy crazy but so does a idiosyncratic aspects to your data center for example here you've got a whole rack where VLAN 100 doesn't show up and likewise on rack three you don't have VLAN 200 so now what happens when Nova launches a VM on rack three on rack three and it wants to connect to segment - it can't because it's none of these hosts have access to VLAN - so it's limits there in which Nova can schedule virtual machines so what that you can solve that problem you can trunk VLANs but now your problem is getting bigger and you can also solve that problem by saying well I'm going to let my I'm going to let my machine launch here and then I'm going to leave it up to the user to choose which provider network to run oh you can connect the segment one or segment two and then the user has to know which one is is and which may have access to and introduced is a great complexity confusion and this for large operators this became an issue that they wanted to address and what they wanted to do was essentially build one single logical provider network that overcame all these problems the single large broadcast domain the fault of isolation zone and user confusion and large broadcast domains so that resulted in on what are called routing provider networks and what you see here that same diagram except now those top Iraq switches are now routers ok and what this means is that every one of these segments the red segments and the recent segments are actually separate layer two segments that are routed between one another and now what you can do is you can take the same the same two VLANs that we had before well now I can call them one logical network again terrifical multi segments one and what that does is it logically joined these two layer two segments into one larger layer 3 network now what that actually does is it takes that large layer 2 that we talked about before and actually except separated into 1 2 3 4 separate layer 2 networks and that's kind of the problem that router provider network solves it takes those four segments and teaches them together logically for the end user to just grab a network and it's going to be able to pop up on that whatever networks happens to be plumbed to that host so you can avoid as well you split the domains so that collapses down to a much lower number the bulk propagation is is isolated your blast radius if you will is much slower you don't have to trunk your VLANs so those are all problems that they wanted to solve you have the one logical network but there's a lot there's some drawback to that you actually need DHCP on segment because GPL means doesn't cross router boundaries with anna proxy but anyway there was a problem that ratted provider networks actually solved and the end result is the user is just attached to multi segment watch any questions on that we will do an example here so here is the actual command that you would use to create a provider network and it's here's the command that we normally use to create a normal layer to provide an openstack network create shared network provider physical network named provider one so on and so forth now that would get now that that's now replaced by a different command where you now have the ability to provide a provider segment ID and then a provider segment named Jake is multi segment one and that results in this output here you'll see we have the name of this network this logical router network is called multi segment one and then what I can do is it will want to created it I can now get the I set an ID and then once I get the ID I can actually add a subnet to it look just like I would a normal OpenStack networking created Network then you add a subnet to diapies to it and here even as I said there are four separate layer two segments I have to add a subnet for each of those segments and you'll see here in this command here so I'm going to take a subnet I'm going to create a subnet on the multi networks multi segment one network and going to specify subnet range juice and I'm going to do that for each of these blocks so going back to the previous example here I've got back a / 16 depends on 124 and a / 24 106 1 X 2 2 14 / 20 so now I'm going to build subnets from that or add something this to that would you see here I've taken that / 16 and I just quit it it's to / 17 here the / 17 0.17 sorry 0 - 0 17 and then 1 2807 6 look better half and likewise for the other segments the 2/24 I split it into two / 25 I had 0 / 25 and 128 525 so I split that equally across those two and now I've got the segment's the addresses and gateways all sort of you know Shazam it that all works this how it's done so now then the user can launch a VM just like they normally would except when they specify the network they're specifying the logical network that includes all of those segments the user doesn't know that one the VLAN is getting an IP address on separate networks it just knows when I go to multi segment one I'm going to get out for the public internet that's professor routed room in stick o toejam so that's that's what router broaden networks do for you and for large operators that's a really big problem because they're both in large clusters they're running provider networks is a very popular an important way to connect virtual machines the layer 2 networks we're getting much too big and it's very fragile a lot of there cause analysis wound up you know pointing to the side of the VLANs as being the root of the problem not to mention the confusion that is introduced to users so that's what router provider networks do and that's the problem that they saw and how it's done so I'm going to pause here for a minute and see if there any questions and I'm going to go to another use case system video and replace the top effects which by router of a physical router or virtual road well most top rack switches are multi-layer switches so they can they can wrap just as easily as a switch so they actually did it exactly exactly the right thing okay so there's no physical change there might be you might have a different you know different software running on it but you can run up a segment to the top iraq and user have a virtual interface with a gateway on top of rack without yourself I think I went B then from I mean event I'm no demon talking means you put more than one VLAN IDs on the same Ethernet wire so say Nick is listening to different VLAN tags so tag with 100 type of 200 they shrunk together on the same interface to zero not sure there so like last step would like to divide that address space between two racks right what if you have gotta add another rack would you yeah yeah so does complaining there's absolutely some planning involved and you're sort of touching on to the next use case so what you just what I've described here is a very static environment where I have set up these environments where I split you can see down the bottom here the 2/1 5 and this required static provisioning of those segments on that router the people under that right they use a route distribution protocols all automated you use BGP to distribute routes and so forth or magically and sort of do that um that is not part of the current release of router provider networks however to do what you ask so let's say I now want to add a third rack and I want to blend I want to split this sly 17 into 2/18 I'd have to go back provision that set the route and you know believe the PM's and do all the things that associated with migrating VMs that would be hung on that segment so a pretty disruptive change so any my example sorry am i running in hundreds no no I ok yeah question yeah so how does how does the gonna game gets instantiated in graph one such example how does the Nova hopes the hypervisor what does that glue which well maybe someone in the audience today it's not better than the anybody I thought for something Kevin like here tonight but I guess not did you configure the network side declared a multi segment one bit right so how does it how do you configure Nova yeah so there's an ECB agent running on each of these segments and that DHCP is configured with this block okay so when what would know the launches there's all of those that I can run exactly with the bootstrapping mechanism and forgotten it but basically when VM wakes up and it basically does a DHCP request on this segment and it just says I need an IP and the HCP replies with an IP that lived on this segment and Noble manages that somehow I don't anybody help out here so the segment idea is summer go forward ah so yeah you'll see here in the the there's a there's a whole collection of IDs and instances so the whole thing multi second one has an ID and then each segment has a segment ID and then each subnet within that segment has his ID and official his a nomenclature is really overloaded and mind-boggling it is reflective so I was hoping not to get into zones everything else we could to stay a little bit higher level time it was another question here also moving at addresses so let me go to a different example and then we can maybe come back to some other questions so yeah what drew me into this whole area was not the use case that I just described the use case that drew me into this whole area was one operator that I'm working with right now is using open to that but they don't employ these very very large VLANs that are running too big their use case is the opposite the situation they have is they have a very tiny viewing / 22 that times its imperfect sources about my freedom so the thousand IDs here and um they also had trouble with their layer 2 technology so what they're doing is they're migrating your whole cluster over to a spine leaf routed design so now they they just do not have the ability to run this as a VLAN anymore because they're running routes to the host they're running layer 3 not only to the top rack or the runnings of hosts so if you use that scenario that I described earlier using for routing provider networks going back to your exact example where you have to split the segment in advance take that tiny block on the flash 22 and split up again tables or cross for racks then further and put that up against all of the different hosts you wind up with the tiny tiny tiny little blocks that really constrains the scheduling flexibility so that was a problem that this operator was facing in there they were trying to figure out can routed provider network solve this problem so the scenario that I just started with radical fighter networks really constrained the scheduling because imagine a scenario where you had 32 hosts here you literally would only be able to have honor 28 256 VM in a whole rack and then you'd run out that problem that's whether that problem does exist in the previous example but they just airs out and in order to avoid the airing out you have these very large blocks so you truncate it in the head room that's a new feature coming down that book we need what you don't have all those IPS you you have to live within the constraints of that where you might have a rack that could only have a hundred vm which obviously is a problem so this is a similar use case of the grounded provider networks but the weight router provider networks are or built right now it doesn't handle this use case very well because ratifying Network sort of presumes an abundance of IP addresses so I'm gonna wrap up here by talking about this other scenario okay right so here's me here's example so they're taking this single flat 22 and they're going through a spine leaf design where it's not where they will they want all of these hosts to access some data center resources here and here's my example I'm taking the slash 22 and as I said I would I would put that up as a 4/24 and as I had 32 hosts I would split that across 32 hosts which actually left me with only a 529 on each host which only gave me meet IP addresses per host okay and that actually might be workable but it's um it severely limits Nova's flexibility so for example as I said in airs out winner when it runs out you might want more flexibility there so the way you overcome that is you give this a slash you know like 17 or something but sometimes you don't have those like users even in the data center 1918 addresses are still running out so you're basically trading off the stands no but election scheduling flexibility across all those available to the end versus how many IP addresses you have to dedicate to this so there's a very direct trade-off between the two okay so the way you are the way unique the way you can solve this is what's going to wrap up here talk about what I'm more involved with this Ramana project what you need to be able to do is to provide the flexibility by dynamically updating the routes and not statically provisioning in this way on a rack or on a host but just allocating IP addresses as as of the end come up you just create a route and inject it into the fabric and let the network through it's tough to distribute those routes so that's sort of the better way to do this it actually doesn't even use the practical writer Network feature of open sex actually just use the standard provider network feature hoping station buddy take the advantage of route distribution and in the Romano software as I and I will show you in a minute and then finally here so okay Romana this is a project I've been involved with sort of Alana dial it's a network and security automation solution we've been really involved with a Bearnaise but we're coming back to OpenStack as we've been drawn back into it if you want more details you can go into that and I'm just looking at the time I want to wrap it up here so in an open back environment Romana would live on Neutron oh it's got some software that manages IP addresses and routes of an IP address management plug-in using the IP an api mlc driver there's an agent that runs on the compute nodes so with that it's um now going back to my earlier diagram about the bridge network versus the robinette ones you'll see there's no bridges here here's the virtual machines that get launched on the hosts and you're there's no bridge this is actually a route router on the on the note itself so you'll see here the default route is to the top of rack so here's the duck 192 go one that's the gateway on the top of rack and in each case these the three hosts on the same rack so they all share the same gateway on the top of rack and then each of these virtual machines get an IP address on the block that is assigned as needed dynamically to to the hope so for example a block would be the fine here in the two machines which would get I peed out of that block that 5:29 but they're not statically for Beijing they're dynamically provisioned and what you see here is the agent then announces the agent peers with the top Iraq and then announces that route so the infrastructure can learn how to reach out of all those endpoints and instead of money ok so now back to that same example with Diana route distribution you can you can um will indicate the okay so in this case I set it up so you actually do split the 22 inches 4/20 fours but you don't necessarily have to do that but the point here is that you can now schedule p.m. to any note anywhere and then the software is going to fetch an IP and then announce that route to the host and it's going to be propagated up to the infrastructure to maintain that reach ability so what that basically allows you to do take that slash way to and populate it however you however it is necessary across all the racks and across the whole to be giving you the flexibility that you need without wasting any addresses and that I think we're kind of discussed to my time here and some 30 on the dot so any other questions before I pass om so I'm struggling what's different about Ramana versus calico over controverse uses PGP is distribution without distribution mechanism to same exact thing in front well two things open contrail uses an MPLS overlay so there what is the variations of that but it could be a plan for MPLS and then use bgp or XMPP to distribute the router president so roughly speaking local contrast has an overlay calico uses BGP like just I showed you there but what it doesn't have is the IP address man so in a layer two scenario you don't need a web distributions with Vermont because it's all later to the rap gets injected through agent directly and though rap distribution sister so the difference is you're with calico you require bgp with ramona intelligent IP address management precludes the need for rap distribution under those a box for those topologies and are other reasons other differences so what is what is the age no not just loosely luncheon Asian yeah I don't want to get a no visit I plug in for kubernetes and as our agent and there's a bird agent if you're running BGP and the missiles yep agent a journalist yeah so it's sort of we haven't released this yet so it's still it's in use but it's not packaged and ready in my decision so yeah so what I was describing is actually version 2 that includes a topology aware IP address management this is important because depending on where that IP address lives in that hierarchy you in order to minimize the route distribution you don't want to put a random IP address there because that you're going to inject host routes that have to be propagated all the place and Lahti and Machado closer bad is not insignificant so having IP address management that is aware of the topology that knows if you're going to launch a VM on that node it has to be topology aware so it pulls an address out of a block within a route that already exists actually is again getting back to the activity the right distribution activity is demonstrable e lower with topology where any members so hey but if we have not released I said this is soon I thought it going to be ready tonight I thought I'd other them over we just parney moving yeah so floating I t's don't um don't have the same meaning in with roughly provider networks because the assumption is that the provider network would already have an at Gateway someplace else so you don't have to use neutrons to give you the floating RVT the only need the floating ID when you're using a tenant network to get off the tenant network into the data center so floating IDs are really not helpful yeah so a couple of questions of you you why did Olivia lost answer that there is only a pretend network at all and this yeah but you outdoors under still may need because I want entire head of networks and that's a totally orthogonal so can either in that case any given instance would have to nix one on the ten network wanting to provide a network a being of the end yes correct right so then a developer would have to manage a local routing table organism to e curse the VM if you were right well your mind sentence correctly look you have basic I would have on I need maintenance wouldn't I be in a provider network but whether it routed or not and I would have a NIC with an ID in attention but out of the gate I believe stuff going I have more than one tenant central cannon project I have to manage my infant's routing table locally to egress corrected with this right I can't do default balance you can use polymer strongly yeah I think because you've got Neutron router involved two that means one of infants well if an instant test well I'm sorry I have to Nixon Grantham then there's only two router right well whether it when they connected to the two unit for the United you two bridges or whatever but the points from perspective on instance I need to know which news originated the packet from so how would I make that decision in this game I don't have any please ask to do it over the other failure if both are valid that the VM will have to decide the local hosts would have this that right based on what information well how I chose the way I make a writing decision I imagine prefix Riker green bitterness element so if I have a single Nick normally I would have an equal drop which is 0 0 right so basically anything matches boom magically Gerson's that make that have the people rock I have to knit you can also have to have about default route you can't have any details but it but they don't golden would you say direction right so if here I have only a handful of prefixes available for my seventh make Yemen thick and here I have direct ie the internet I can't make a decision on the VM intelligently I don't have that information on the on the right you have to program the routing yeah I think this is a question it was beyond over sectors say I think that's the word disingenuous organizational well it's an implementation problem all the time right so when the developer Association into the ministry he would have to create persistent routes based on piece of information that was supposedly not available for now right yeah so they're all right they're all absolutely right I get my only response is that operator could choose to use both provider intended networks and they introduce all that complexity right and other operators won't have any people dishonorably so they might choose your that's why I thought you alluded to basically lack of tenant networks because it's operationally complex but I wanted to kind of ask a different question that was this almost gonna be a lot response well the different questions I want to understand what is dealing with what each communal is appearing with the top of rack router over flush that you want so now I have two marriage or to be a very on average about the reflash that you want for Tommy around that that that's what the Romano software does that could be me to be a member for tech community as well and I can be sick all below the below the surface okay so that's one of the reasons so with Romana need to run on a couple racks which is all good then w limiting need to operating system operating systems that support be GPM numbers well the height of the peer relationship so that is one yes so we did not wanted respect any those bills bill time you're exactly right you can't be supported on the end for the other end of the wire rusher I mean if you're in / 31 you could also grab the IP at the top Iraq when you're deploying in river there's going to be a lot of flesh 31 yeah the 30 flesh didn't want to perhaps see if I'm managing a haunted racks and that set up stop massive amount of bypass yeah but if you do it on knowing if you Facebook row or whatever you can calculate recently we'll have some time I think after learning Fox laughter little over what you can nicely we respond that that because the data center design choices that they decided to take layer 3 isn't most buying that choice they embrace that burden so depending on the burden of managing those issues let's think also the tourism section successful

Channel: Cloud Native Containers

Views: 799

Rating: 5 out of 5

Keywords: OpenStack, Networking, L3, Layer 3, Provider Network, Romana, Routed Provider Network, BGP

Id: mcSKQxzRnsU

Length: 37min 22sec (2242 seconds)

Published: Mon Jun 05 2017