WISP Design: Using eBGP and OSPF transit fabric for traffic engineering

good afternoon so for those of you don't know me my name is Kevin Myers I'm a network architect with IP architects and today we're going to talk a little bit about whis design so know there's a lot of wisps in here we talk to a lot of wisps I work with a lot of widths and one of the things we always talk about is traffic engineering how do I manage my traffic through my towers and through my width so we're going to talk about that a little bit today with a new design that we've come up with so let me tell you a little bit about myself before we we jump into this just my background I've spent about 19 years now in networking I started in the late 90s and I think one of the things that I'm most proud of is we've designed and built networks on six continents and actually not so much because of the network's just because you get to see and interact with cultures and see what the networks do it's really neat to work on all these different countries and all these different cultures and see you know how the networks that you build affect different people so it's something that I think is really neat I'm also a mikrotik certified trainer and hold-down mikrotik Cisco and Microsoft certifications and one of the other things that I know a lot of you through that I'm really passionate about is community involvement in networking so spend a lot of time blogging writing going on podcast different things to kind of talk about networking and talk about some of the issues in networking and some of the things that we're doing so probably one of the most notable I was really happy to be a part of is the packet pushers if you ever watched the packet pushers they're a great group even banks and Greg Farrow and drew Connery Murray all those guys actually got a chance to go on one of their shows a couple months ago and talk about white box networking and a networking field day I'm also group contributor in this talk and router West and few others on Facebook so if you're on there feel free to hit me up there I was a delegate for network field day 14 in Silicon Valley in January earlier this year where we went into word a bunch of a bunch of big tech companies and talked about next generation of networking I'm very active in the micro tech forum so you can also look for me there and also on another podcast called network collective so you can find all those on the web my little pointer is not working today so before we go to the presentation I do want to tell you a little bit about my company and about IP architects what we do is essentially revolves around consulting so we have a background in network engineering in service provider data center and in cloud data center so data center falling between enterprise as well as cloud service providers for data centers and we do global consulting we do it all around the world we've got offices on in three countries now we've got offices in Montreal two offices in the US and most recently we've opened office down in South America denim Venezuela to do consulting down there we do fully manage networks we do network monitoring and we also do load testing and development so one of the more interesting things that people come to us for for an example we had a large Silicon Valley company come to us and tell us they wanted to be able to put a hundred thousand connections of a specific type through a mikrotik terminating and we actually figured out a way to do that for them so we were able to do load testing in our lab and kind of build some really unique solutions for some interesting problems so let's talk about the goal of this presentation and this is one of the things to kind of help you give you an idea of why we're talking about this and what this is all about and hopefully when you walk away from this you're going to have a few things nailed down one of the first things I'm going to talk about is if you use OSPF or maybe use static routes or something other than BGP for routing your wisp some of the limitations that you hit this is actually point one of most common reasons people come to us for consulting is they've hit a wall in the routing they they're trying to move traffic around they're trying to do something and they've compared to complete wall notice behalf and once you grow to a certain size you will outgrow OSPF it's almost a given so we're gonna talk a little bit with the limitations in OSPF another non BGP routing protocols so that when we talk about ebgp you understand why this is important you know when would I use this we're also going to talk about something called an OSPF transit fabric this is something that we developed out of our office in Mississippi for a client that needed to solve a very specific problem we ended up publishing it on our blog which is subarray 51 net and it allows us to use all the bandwidth between two towers to create the fabric between the two towers to leverage all the available bandwidth so we're going to talk about that a little bit and then we're also going to talk about at the end the benefits of using ebgp and communities to build a scalable framework for managing your traffic you know how do I manage my traffic you know other than just going to each tower and putting on a bunch of manual custom rules how do I build an architecture that allows me to pick an exact our path through my network by subnet and manage traffic that way so we're going to talk about that so I told you we're going to before we get into the solution we're going to talk about the problems because I think it's really important when you when you're doing network design you're building networks it's really important to understand what the problems are and what problem you're trying to solve and something that we do a lot since we work in architecture a lot but we always say there's some kind of a business driver that we start with it leads to a technical problem that we're trying to solve so in this case we're going to talk about the problems that you run into when using what we call an IG P an IG P is an interior gateway protocol it is a non EGP protocol so examples of this would be OSPF and rip are the only two that are really implemented in mikrotik you also got e IG R P you've got is is and there's actually a few more out there that are working on development in the IETF mostly they're designed with reach ability so we're going to talk about you to legalizing OSPF for all forwarding of traffic so as the network grows this creates suboptimal traffic flows you know what do we mean by that so let's take a look at a very basic example of here's a whisk that we built kind of a ring architecture and we've got links on the northern path here or there's 750 Meg and then we've got a southern path that uses 500 beg links so if we let OSPF because the radios are what we call sub rate Ethernet it's the speed of the physical port on the radio is not equal to the speed that you can actually put down the path then we have to make sure we adjust our SPF cost if you leave us pay off cost by the default out of the box then it's going to take the slower path simply because there's fewer routers in that path it takes the default cost for Gigabit Ethernet which is 10 and it's going to take the shortest path which happens to be the slowest path so how would we go about fixing that and OSPF so now we say we're going to get better or worse so we can modify the cost on the lower path in OSPF and say okay I'm going to make the cost and set at 10 and make it 15 and 15 and 15 and make it 5 5 and 5 on the top so that that's a worse path so it's a lower cost to get on the path and that's our new best path up the northern faster side everything's great right well now it's worse because we just added a tower and so now we added this tower down here hanging off the tower there at the very front and it has a default cost of 10 well when it goes into that tower then you just added more load to that northern path and maybe because that's the path you have nothing traveling that southern path and maybe you really needed to use that but because you were jacking around with those PF cost everything is loaded on that that side there so how are we going to solve that problem so now like oh I want to use both paths so now we're going to SPF and we're going to we're going to tune that cost to say ok I'm going to go 15 15 and 15 which gets me to 60 on that top path and then I'm going to go to 20 and 20 which gets me to 60 so now from the tower where we started the customers hanging off of I equal pass back so now we can leverage something called ecmp if you've never worked with the ecmp it's a mechanism in OSPF that allows you to select more than one path and load-balanced down more than one path for traffic headed back to a destination so now I can load balanced traffic across both paths by setting my OSPF cost to the same so everything's great right maybe not now we've got to add another Tower out of there and now we've got overloaded paths on both sides but 250 Meg that we can't use on the upper side because we've set equal cost down each side so because you've got 250 Meg there you're going to start dropping traffic because the flows hash equally the network thinks they're equal bandwidth so you're going to be seriously dropping traffic on that side and you may end up getting overloaded on that side as well so some of the solutions that you may get into if you have OSPF is you may get into static routing or you may get into policy routing and then the problem you have with those is not only do they not scale but you know oftentimes they break and fail over you know there's a lot of times you put static routes in a lot of times you put policy routing in that it just doesn't failover very well and then there's a reason for that they're really policy ratings designed to solve very specific problems that you have it's really not designed to replace a routing protocol so what we're going to look at if we get into the OSPF transit fabric slides and then getting into the ebdb slides is how do we take a routing protocol and solve these problems that is designed to solve these problems and I think one of the most important things to understand about OSPF is OSPF was never designed to be a policy protocol it's designed to be a reach ability protocol it's designed to say I want to map a path and see many paths that I can get to certain places and then I'm a map and understand those paths but it was really never designed to say okay now how do i leverage those paths and how do I use that that's really what bgp is for which is why we're going to get into that and talk a little bit about it so here's the second problem that we have we get into SPF the first one was dealing with policy and managing where am I going to steer my traffic to the second problem you get into is the vast majority of lists that come to us for consulting and they want to do routing architecture redesign we'll talk about having active and backup path so we'll say ok I've got this one path and I want to be active and I want my 500 Meg to be backup so my effective bandwidth between any two towers there is 750 Meg and in the top scenario or alternatively very commonly done is I'm going to take one of those links and make it only for downstream you probably so SPF cost to do this and then I'm going to do a different OSPF cost in the other direction and I'm going to make that 500 bag for my upstream traffic that's a very common split that we see in whips so the problem comes in is you think of the the applications that flow across ISP networks today it's you know it used to be we were dealing mostly with web pages need a little bit of video and then you had music well now it's Netflix and it's YouTube and it's Hulu when you're dealing with really high volume you know bandwidth applications that you're trying to compete with a wireline carrier you're trying to compete with the Google Fiber you need every single bit of bandwidth you can get so if you've got 500 mega bandwidth sitting on tap that you want to use why wouldn't you use it and that's kind of the problem that it wisp came to us with and we were like well ok let's figure out a way to use that so in going further before we get into the OSPF transit fabric in the in the in the upper half we have an overloaded active link and an unused backup and then in the bottom we have a link that's unused for downstream and then overloaded on downstream so again you can see we're pinning traffic to different paths for very specific purposes gets you into trouble if you need to leverage that bandwidth so OSPF alone doesn't work and i say it doesn't work Oh SPF we usually get you to a certain size and a certain scale but what you'll find is as you get into more towers you may start out with one tower and then you get to ten towers maybe you get to 20 towers and then when you start getting into like maybe 50 and 60 towers or 100 plus towers you have a topology that starts to happen in your wisk where you'll have rings and you'll have a partial matching of all these things that start to form animos PF starts to break down pretty badly the other thing I really didn't put in this presentation that we've seen in production that's a major issue is typically OSPF convergence will get to the point where when you get to about two or three thousand routes in your OSPF table and the single OSPF area with mikrotik at that point typically going to get to the point where convergence is so slow that the protocol is unusable so and there's a reason for that Oh SPF is forced to learn the routes of every router that's participating in that area it's why it's a link state protocol and so when you have to do that then that really slows down your convergence whereas BGP doesn't attempt to learn every path it really only wants to learn about the prefixes and the policies and so the solution we're going to show you actually blends OSPF and let's so SPF do two things that it's good at and then we let BGP do the things it's good at and it's kind of a hybrid approach so and the thing is with OSPF again every time you do a workaround on OSPF something else is going to come up you're going to get everything set just right and then you're going to add another tower and then all the work you've done is going to be undone again because the OSPF is used for something really wasn't designed for and so I mentioned this before OS PF is a reach ability protocol but it's not a policy protocol the function of link state routing is to map the available paths and speeds to prefixes and so let's look at some potential solutions now we've looked at the problems and limitations that some of you may our have already hit some of you may not be there yet you may be a you know a couple three years away from this but you'll know it when you get to that point because things are going to start really slowing down you really start to see some serious when you have a pass/fail and you and you get big enough we've seen in production networks in fact there's actually one gentleman here that we solve this problem for he was going into 30 minute convergence times it took 30 minutes for his network routing table to converge across I think it was several hundreds hours so when you get to that point you want to look at using something like a transit fabric and you may do an IV TPAs so the first bullet point and IB T pas this is something we use currently it's a design that works very well is something that we've taken from the wireline carrier world where you form an eye BGP autonomous system over OSPF so you take your transit subnets and you loop back you advertise them into SPF and then you appear on your loop back to form an IV GPS that's kind of outside the scope of what we're going to talk about today I just want to mention it because that is probably most common way we solve this problem now and the way we do it and then what we're going to show you is kind of the next gen the next gen design but if you want to know more about IV GPS I think I did a presentation on that the 2013 mom I believes you can go look up there's a whole deep dive on how you build an AI Beach pas on the presentation in 2013 st. Louis so the second bullet point combining ebgp and the OSPF transit fabric it allows for total control of the tower path and you can use all of the bandwidth between any two towers so you think about that for a second you can tell a single subnet the exact tower path you want it to go down doesn't matter if it's preferred what the routing protocol says you can tell exactly where you want it to go and what's even better is let's say you have four sectors at a tower and you want you have four different paths and you want each of those subnets to travel a different path back to your pop and back to your core you can absolutely do that and again it's not not all or nothing so with this design while it's something that you can leverage across your entire network you could put it in a section of your network and then start migrating towards it in pieces so while we're showing it across an entire network it certainly is not something that you would have to overlay all at once in one Big Bang you could bring it in two pieces of the network and then keep rolling it out all right so let's talk about what an OSPF transit fabric actually is that's the name we gave to it simply because transit meaning we're trying to establish connectivity between two points doing two towers the subnets and we call a fabric because in most if you in the data center world most fabrics are defined as a large number of equal cost multi path hops and that's what we have here so we went a number of ways to tackle this I think we went through I don't know how many Keurig pods we went through because we've spent about three days on this at the office and design before we finally came up with something that worked and what we discovered is OSPF didn't necessarily need to know the speed of the radios but we did need to find a way to tell the routing protocol how to balance the traffic to the different speeds that we had in the towers so if we let the VLANs inherit their default cost which is 10 and map 6 VLANs on the northern path and to VLANs on the southern path where the traffic comes in free cmp it balances by source and destination by flow it's not my packet so if you have a conversation to Google and it will take that and map it down one path and then if you go from your computer to Facebook it will hash down another path and a round-robin like that back and forth so what we found is by establishing a ratio of traffic by mapping more VLANs on one side and a few lines on the other it exactly mirrored the traffic capacity of those radios so it would be able to use every bit of bandwidth in one direction and every bit of bandwidth in the other direction so that you can kind of just glue all those together into a single set of links so six VLANs on the top we did this in a six to two ratio which is really a three to one if you break it down but that was the what we did was we divided the bandwidth we divided the bandwidth on the bottom into the top and then came with the closest ratio to match Allah in the VLAN to get our traffic ratio and so we realized that we needed sixty lines on the top and two VLANs on the bottom now because that 62 fraction breaks down to three to one you could do it with three VLANs in one VLAN but what we found is sometimes you want to add more VLANs because the more you add the more granular the traffic flow and the balancing gets so if you add more VLANs it tends to get a little bit better on balancing the traffic out and then conversely on the southern path we have two VLANs to make up the to ratio to bring bring that traffic and draw that traffic down on the southern path okay so here's this is actually this is actually we pulled this out of production so this is when we first put this on you can see when we talk about equal cost multi path there's our default route and as you can see it has eight gateways it knows eight ways to get to the default route so it's going to hash equally down all those paths let's generate the traffic balancing and then actually looking at the live traffic in this case we had a number I think we like seven or eight radios between the towers and so we did that with the we egghead backhaul six and seven and the test here and we put our two VLANs there and we put our six VLANs down there and then if you look at the aggregate traffic if you add that all up you will see 22.8 mags on backhaul of seven and 6.2 mega on backhaul oh six and if you divide six to into twenty to eight you get pretty close to six to two or three to one in your ratio so now I really want to know is to have transit fabric but how does it scale and that is a question we got a lot when we first proposed this actually actually we first proposed it everybody said it wouldn't work then we put in depression it worked and they said oh how does it scale so we had to kind of think about that because it's a really legitimate question because bringing a lot of OSPF routes into there and doing it between showers if I wanted to take that across two hundred towers left a lot of VLANs and that's a lot of OSPF routes and that's where ebgp kind of came into it we started bringing a BGP into the design and so I think wisps that are under fifty towers could probably use this as is without needing to layer ebgp on top unless you want to DB GP for the traffic engineering piece now I'm speaking strictly of how large you can get with OSPF without slowing your convergence down so I think if you're under fifty towers you could probably put the same without learning be BGP on top of it if you're over fifty towers we have really complex traffic management requirements I would probably go ahead and couple it with ebgp to get the best of both worlds in fact after we leave here at the mum we have one of the clients we rolled this into production for they really want to get into the ebgp design and so we're actually going to be putting that into production for the first time here in about two weeks and so ebgp provides a way to limit at OSPF will get in this in the drawings it limits OSPF to just between the two towers so instead of all those routes like flooding around your network no SPF they actually just stay contained between the two towers because we're using a BGP and we'll see how that works here in just a second okay so let's talk about ebgp as a traffic engineering mechanism we did the OSPF transit fabric now let's talk about a BGP and see why this is helpful and what this does for us so one of the major benefits of ebgp is the ability to definitively select the next hop by matching attribute so one of the beautiful things about BGP is it has a series that is about eleven steps that it goes through called the best path algorithm that if you don't influence it in any way it has about eleven different options to evaluate to figure out what is the best next hop but as an administrator you can override it and tell it to go anywhere based on a whole number of things so BGP communities or what we're going to talk about today and we'll get it a little bit into what that is and how it's helpful to you and so here we have a whisk that is five towers arranged in a ring we have one pop over on Tower one it's going out to the Internet and on the path what we're going to call the southern tower path we've got multiple links between the towers so we got about a nine hundred Meg's shot between one and two nine hundred mega - and three and then we have dual links of 750 and 500 between Tower one five four and three so effectively to go from Tower one to tower three I only have 900 Meg on the Northern Pass and I have about 1.2 gig on the southern path so how does V V GB differ from ibgp so one of the one of the common misconceptions about ebgp is that ebgp is for the internet which is really not true ebgp simply means if you're going to establish a peering to an autonomous system that is not the same autonomous system as what you've set that is an e bgp peering because it's assumed bgp was first developed it was assumed that a different autonomous system and somebody else owned that and controlled that and that was it was how that was done that kind of use case for this if you want to go look at the use case for where we drew this design Microsoft first proposed this in their data center several years ago they were having a problem scaling in ibgp because they were doing hundreds of thousands of servers and so the only way they could find that they could scale the way they needed to and build the policies they needed to even just within the data center was to use a different autonomous system in every rack and so they literally were doing BGP down to the rack doing a different on the system and so that's they were kind of the first ones to do this use case and now it's blown up into a bunch of different things so we adapted that to this use case so you can see we've got five economist systems here these are all in what's called the private range that's reserved by I Anna and we've got sixty five thousand one at our 165 thousand two it's our two sixty five thousand three and three and sixty-five thousand four to our four sixty five thousand five at tower five now I put our AR is up here if you look right there that just means your regional internet registry so for those of you that are here in North America it's going to be Erin if you're from Europe it's going to be right and there's a whole bunch of other Lac Nick and they're a bunch of other regional registries you would get a publicly registered autonomous system number from them those would go in your BGP edge routers and so what you can do is because these are private autonomous systems you can set a policy at your BGP edge router to strip all these private systems out of the S path so that as it advertises to your upstream all you see is a single public AF so let's look at I try to make this as big as possible because I'm really really guilty of doing like tech presentations I'm enjoy tiny nobody can read it so I try to make this as big as I possibly could so that you could see it but but here's where it all comes together here's where we combine the OSPF transit fabric and ebgp so the challenge we have is if you're doing a BGP peering you normally are going to do it on a directly connected subnet you're going to have you know a cable or something stretched between two routers and you're going to have it directly connected subnet between them and you'll do a peering and that's what we have in this case because we have two radios and we have our OSPF transit fabric so here I put three VLANs on the northern path and I put two VLANs on southern path here and so what I want to do is I need to establish an e bgp peering across those ebgp has a function that's supported mikrotik called multi-hop so that if the router that you're peering to is not directly connected to you you can use another routing protocol to advertise the period dress in this case we're using OSPF so everything in Blue is related to the OSPF transit fabric so if you see the blue loop backs we have our neighborhoods agencies formed in all those VLANs for our to build our transit fabric and use all that bandwidth and then we have a loopback called Tower one to Tower five and Tower five to Tower one and so the point of that is we advertise those blue backs into OSPF because bgp can then use those IP addresses to peer to each other so that these towers can form an e BGP peering it also has the benefit of once that loopback has learned over OSPF it sets all those next tops for the transit fabric now we go into our red address here these loop backs that we put on the router they're advertised in BGP and so when you go to the next router downstream that if we were to jump over to tower four which isn't in this picture the only addresses we would ever learn we wouldn't learn any of the stuff in blue none of those OSPF subnets or those loop backs would get learned by the next router you would just learn the 10th 199 1.1 and 199 dot 1.5 so this is the answer to the scale problem the ebgp relationship keeps all of that OSPF complexity in all those VLANs and all those subnets slowly contained between those two towers and never gets advertised outside of those two towers and it works because ebgp automatically rewrites the next top so let's talk a little bit about bgp communities BGP communities are a field and a BGP route advertisement that you can use to take some kind of an action or give you some kind of identification about a route tag with that community number they can also be used for identification it's a 32 bit format where the first 16 bits is typically set to the ASN of the network that you're working on and the last 16 bits are defined by the operator that's kind of a loose assignment that you really don't have to do it that way that's just kind of the standard they recommend that you do so what we did here is we did a couple things we built five what are called informational communities so we bill we we decided that our public a s is going to be 1 1 1 1 1 and the first community was going to match tower 1 and then 1 1 1 1 2 is tower 2 and so on all the way down to tower 5 and the sole purpose of those is just to identify where the Africa is coming from so before we go to the traffic engineering think about helpful it would be if you could go into a routing table if you've got a couple hundred towers and you've got tons of routes to be able to go in and look at the route in the routing table and be like oh that came from Tower 6700 that's capable Tower 89 you can kind of build some intelligence that you can actually later then use for traffic engineering and all kinds of stuff into your network to identify where it's coming from then we're going to identify some traffic paths we're going to say okay anything tagged with one one one one 200 is going to take the northern tower path Tower three two one anything that's 1 1 1 201 is going to take the southern tower path Tower 3 Tower 4 antara 5 and Tower 1 you could do incredibly complex variations on this if you had a if you had a tower that was an aggregation point and it had a bunch of towers hung under it let's say you had 4 towers hung under one Tower you could say I want these 2 towers to go go up the aggregation point and go left and I want the other two towers hung under it to go up to the aggregation point and go right and so you can build an almost infinite number of traffic engineering communities that you as the administrators set and so again this is all arbitrary it's something that you define so changing the next top how do we do this and this is a something that gets a little bit complicated will actually have a hands-on demo of this so that you can get a better idea of this so don't feel like you have to digest all of this just from the diagram because we actually have some stuff that we can get deeper into it and show it in in real time so the way this works is I don't get my little pointer out here so here's tower 3 we're going to take sector number 1 we designed this last 23 to the first sector of tower 3 which is 10 10 dot 0 2 0 / 23 and we're going to advertise it down both paths in bgp we've got our pairings built and so we're going to advertise it as we advertise it we're going to put two tags on here we're going to put the tower 3 community tag and we're also going to put the tag that we want the southern Tower Path 201 is this path down here so that gets advertised both these towers and I get to learn back here at Tower 1 so it learns it down both paths then Tower 1 has this set here walk over here it's a little bit easier Tower 1 has it set here to say when I receive a route that's tagged with 200 which is the more than tower path set the bgp wait for that pier two three two four six seven if you're not familiar with BGP wait it is the absolute highest priority way to set preference in BGP for traffic it trumps everything else that you can possibly do a s path local preference any of that stuff so the minute you set weight to force traffic out of specific direction than anything that matches that community is automatically going to get sent that way no questions asked same thing down here so that's if we had tagged it in 200 which we didn't we tagged it in 201 but in 201 we said if you match 201 then set the weight of that peering to three two four six seven and these are all preferences if one path goes down then obviously the only path that remains is your best path so it still fails over but in this case because we've done that and we're going to look on the next slide what that actually does for us so so we're going to look at influencing traffic down a single path so by tagging the subnet from Tower three in the community 101 201 we're able to force the traffic along the higher-capacity path leveraging our OSPF transit fabric we can use all twelve hundred Meg's between all those towers and we can force it down this path so even though we have two more routers here and the routing protocol would naturally see it as a longer worse path we can say hey the minute I build out a new sector I can tag exactly which path I wanted to to go down so there's two great things that happen here one not only are you forcing it down a path once you've built all these communities in there in and done so the minute you build a new tower and a new subnet the only thing you have to do to make sure the traffic goes the way you want it is write a very small short route filter tagging it down the path that you want it to go and then you already have all the traffic engineering intelligence built into the network so all the hard work is done you set a tag and it flies the way you want it to fly now we're going to go down two paths so and you could you can go as many as you want with this if you happen to have eight redundant paths back to your core back to a pop you could slice traffic eight ways if you wanted to so we can take we're going we've introduced another subnet here so we've got ten dot 10.0 23 that was our first subnet and that's flowing I've got this in the downstream direction so most ISPs are heavy on downstream and lighter on upstream so we've got it going from the internet back towards the subscriber over the southern path like we talked about the first time but now we said okay I'm going to take the second sector which is 10 10 2.0 / 23 and I don't want to load this southern path I want to load that second sector at Tower 3 down this path so the only thing I have to do now that we've built all that intelligence in is tag it in that community and then it automatically flows down that path and so as you scale out this gives you the opportunity to really kind of manage your traffic in a very fine-grain way that just is absolutely impossible with OSPF ok so we've got to the point where I've got a couple more slides but I want to pause and see what questions that you guys have what you know what questions you have about this this design yes sir you would set it in the correct so for your default route for upstream traffic if you wanted to do traffic engineering in that direction you would do the exact opposite on the originating router so in the lab that I have you would actually set it in the other direction to set wait for the USB like for your for the routes you're not sitting mid for the other direction you're not setting the metric you don't need a metric because weight is the highest weight is the highest bgp attribute that you can set the minute weight is set it trumps all other bgp attributes so you're substituting an advertising metric by using a community that causes weight to be applied at the other end at each tower yes okay yeah at each tower so you're using it at each tower to make a traffic decision point Kim did you find that this method was easier than using an MPLS traffic engineering base I am glad you asked that question so the reason this design exists because there are some limitations in the microchip element ation of traffic engineering so if you are using MPLS and you're using VPN v4 and you want to build traffic engineering tunnels and normally if the next top of if the next top of the traffic engineering tunnel matches the next top of a route like within a berth within a VPN v4 verb it would go into that traffic engineering tunnel unfortunately that configuration is not supported in router OS one of the other limitations you'll find is that the LDP protocol in mikrotik router OS is not set for equal cost multi path so one of the challenges you have is if you want to leverage all that traffic between your towers the minute you put MPLS on to all of that you've suddenly because LDP and the enter less fib is not making the forwarding decision you've basically broken the transit fabric and put all of that traffic down one radio path hey Kevin hey so that kind of leads in the next question about VPLS overlays obviously you can't load it with VPS because of the source and destination IP s so the neck is that the question is you can't do VP LFO and it's because it's about routed subnets through the networks so the first piece of this design we tackled was o step transit fabric like about I guess was about seven months ago about three months ago we tackled this piece we actually have a design in the lab that solved the MPLS problem and the way we're proposing to do it is by if you are going to sell an MPLS service and if you need an MPLS service we're going to look at using another VLAN down each path for MPLS with subnets that are only reserved for MPLS so that as you need to put an MPLS service on top of that then you have a subnet it's solely for MPLS so that non MPLS both traffic that needs to take advantage of all that can go those that path anything it needs to be MPLS will have another set of e lands that you can put it on to you said LDP didn't support ecmp but does RSVP the traffic engineering sub-tab know so in the implementation it does not either is an implementation in India so if you're familiar with cisco juniper brocade you have you absolutely can solve the problem the way that you're suggesting this is actually a very specific response to that limitation of not being able to leverage MPLS traffic engineering to solve that specific problem specifically within router OS in many of our OspA that works we're using auspi to distribute to the public to subnet we've assigned to a customer that of course flows throughout the network sure how do you deal with that in this scenario because your examples are should are showing a hierarchical breakdown how do you how do you deal with advertising a customer subnet as far as right so I signed a splash 32 to a customer huh and it's in the next 30 to maybe on a completely different Tower okay um so in that case if you're doing it at a / 32 boundary then you would probably need to look at tagging communities you'd have to build communities that map your specific profile so the communities I use were very generic they're very simple they were we're going to go north and were going to go south but you know ultimately the communities are whatever you want them to be so you know if you wanted to use other bgp attributes to map into the communities you know weight is not the only way to do it it was you know easy enough for us to do in the lab there are a number of bgp attributes and then combinations of those communities that you can aggregate together you know at a point to say I want you to do this so this is kind of a starting point it's really not the end-all be-all but you would have to deal with that at the 5:32 level of assigning communities to treat them in the way that you want any thank you all right any other questions yeah yes sir implemented way thank you would there be a way inside of bgp in order for it to track let's say if you had a heavy snowstorm they would affect the qualm rating of the Lycans link words said of his pushing 700 mega thick it's only able to push full hundred Meg's or BGP could recognize that and also to a different route the answer is is sort of yes so it's a slightly different problem that we're working on solving so when I was out actually mentioned I went to network field day 14 in January out in Silicon Valley one of the shops that we visited was Juniper Networks we visit their global headquarters and they showed us kind of their next gen stuff one of the things they did and we're looking at adapting this into mikrotik is what we call dynamic QoS so they actually had a problem where they were taking they had satellite networks that were dealing with rain fade and the QoS would be needed to expand and contract to match the modulation of the links and so they actually came up with a solution that was able to basically scale QoS to match the modulation so that you know you could you could build your QoS policies appropriately you could expand that then into you probably would want to get into some kind of a controller at that point there's so much complexity there you'd probably want some kind of centralized maybe yeah and that's the end controller or something to take on and manage all that but the answer is yes you could probably rewrite the BGP communities on the fly to match the changing RF conditions you know by using a combination of querying SNMP or an API on the radio to get the modulation dump that into a controller that they can then intelligently you know set communities on the fly to see your traffic where it needs to thank you we have another one could you also just shut off a VLAN in that case so I do you could I offered my qua missus drop exactly the action that you take could be if I could be a number of things you could do to decide exactly how you want to react to that in this case have used the transit fabric you could the only reason I might not do that is because if I if I have an ending QoS architecture and I have links that don't utilize a transit fabric then that wouldn't be unacceptable so I might want to just heaven in QoS and then shrink and expand those policies you know based on some kind of centralized intelligence okay all right okay so we actually built this into gns3 for those of you that are familiar with gns3 and we have it over at our booth so all of you that have a number of questions about it you can actually see this live we're gonna bring it up on the big TV fire up the VM I don't have every single bit of it in there we basically are just showing that we're influencing on one path so you can see how the community reacts but you can kind of get an idea of how the transit fabric works what it looks like what the routing table looks like and what the policies look like actually in the config to do this so come by the booth and we'll fire this thing up and we'll talk about it a little bit and last but not least I think a lot of you've already heard about the BGP party we're going to billiards and grub and network planning we already have whiteboards waiting and that's going to be a tarantula billiards at 6:30 so if you would if you haven't already please RSVP on Facebook go to our Facebook page RSVP if you'd like to go and we hope to see you there thank okay thank you Kevin so this concludes our first day of the mum thank you all for coming we'll be having drinks hopefully earlier than files yes and those of you who are mikrotik certified trainers please come back for trainers update in ten minutes

Channel: MikroTik

Views: 7,090

Rating: 5 out of 5

Keywords: mikrotik, routerboard, routeros, latvia

Id: dFZz2z6RdQY

Length: 40min 13sec (2413 seconds)

Published: Mon May 29 2017