ISP-Network-Design

right so will this stop make a start go one more presentation of what to do and that is the isp network design one it's not really anything to do with bgp or ipv6 or sad but I still think it's useful know it's quite a few of you feel you are starting off new networks how to design things how to interconnect what to do and so forth so I thought it would be useful well I was asked to do this so I thought it'd be useful to put it into the workshop just to be making make it part of we're going to cover so it is not too long that will just quickly go through it so that you've got some good ideas for how to do your infrastructure design and it backs up some of the principles that we've been talking about it and doing the workshop labs when we look at some of the bgp design and so forth so these are the topics we've covered some of them already but we'll look at top topologies and design backbone design addressing we've done we've covered that a couple of times but I'll quickly go through it routing protocols we've got as well we'll come on so security is not a lot being talked about here it's bad security but I'll show you some of the basic things you want to think about other plan management is very important operational considerations as well that's like just operating the network how to do it what accepted good practices are as well so this is in the design presentations folder in the handouts that you got so if you want to follow the slides you can do so the first thing we look at is the point of presence design we've talked about core routers in the first day i speedrun connections distribution access bigger networks of distribution that know size once you have the access routers where you connect your customers to the network border routers connect you to other service providers it's very important that you have a border router in your network so it's separate from any of your backbone service writers where you do all the hosting and services but as I said on Monday some of these functions can be handled by a single router it's not that you have to go to your vendor and buy all these different writers from the start you can start with one writer and as that fills up you can expand it out by another one of the same size and start building a network like that it'd be more scalable and offer better redundancy then trying - just buy a bigger and bigger single box so these are the general allocation of routers that you get in most ISP networks now with the pops you want to try and do some type of standardized design the bigger ISPs will separate the aggregation services currently connection speed the type of customers I talked about contention ratio earlier in multihoming or whether you want to do two to one for high-volume business customers six to one for normal business customers thirty to one for consumers that's typically what is PS trying to do I'd also want to bear in mind the security considerations as well that you want to try and bear those in mind also security is in a high responsibility for ISP and security is not just achieved by putting a fireball at high speed don't have firewalls it's the end science the end users that have firewalls ice peace about moving packets around and the security to nicely does is to make sure that infrastructure is here right so we start off we talked about core routers who have got the two writers in the core with God Blackwell links to other pops so we saw that on Monday typical type of design got to Colorado's if you have other points of presence in your network then you make sure you have redundant links not just one is be services connected like this so it could be your DNS primary second videos your mail servers your mail systems the user you have that FTP web servers any other content service type delivery put them in the Bourdon piece of network don't put your mail and your DNS servers on the same land as your customer service so don't put them on the same place as your customers have a dedicated subnet for these if you have multiple pops in the backbone spread your services around the backbone as well again for redundancy if you're doing hosted services data centers and that type of thing separate Lam separately I mean how you do this it can either be a land hanging off the core routers or you can have dedicated data center Rogers here which then up all the different posted lands sitting behind them it just depends how big the data center effort is going to be and then we have the customers so we've got leased line customers so how do they connect whether it's the traditional TDM leased line or whether it's fiber or point-to-point wireless or or whatever but that's your leased line aggregation layer sitting in here so again it would be a separate business unit perhaps the bigger networks tend to do that and you might have met your Ethernet so again some parts the world you have Metro Ethernet so again you've got the aggregation layering here the metrics and have routers connecting into the network or what else border routers they have a very tight link into the network core so you got your border router here if you've just one pop you'll have two routers if you have more than one pop it's a good idea to have one water here one border somewhere else again for redundancy if this pot breaks the least you can still carry on running your network from elsewhere where the caching is always a good idea whether you do transparent or non transparent cache web cache is quite often go at the border network they can often go down to the customer portion as well as I'll show you in a few slides but when caching content caching is always a good idea because it saves value network operation center Geir depending on the ISP size the bigger ones not has a dedicated connection into the router to the network or you don't want your dog to be sitting somewhere in the company off its land behind a firewall very bad design because if something happens with a firewall or the company aligned you can't run your network so the network operation center is attached directly to the core of the network any issues you talk have direct access to the network course we've got immediate access to the network Pro consumer access while you know the cheaper consumer access it depends you can come in different places connecting directly to the network or or maybe some other part of the network but you know they were all hanging off the core as well so that's a difficult type of thing you go to the really big ISPs they have different business units running these different functions the group different business units running these different functions the smaller providers it tends to be just the same people doing everything but still worth thinking along the lines of how a business going to scale in the future you always want to try and design things to be nice and scalable now the IGP design using wasp yet for example you want let me try one area per pop you don't need to do this much too much where by esaias but try and do aggregation within IGP if it's possible that things like point-to-point link addresses used here we can aggregate those into ibgp into the core rather having lots and lots of little snatch thirties running around in a backwards to be careful with your IGP desire from bgp retro sector so again the two curvatures are the right reflectors for all the other writers and then the part depressants very common design the two quran just are the right reflectors that way you can scale ibgp these two core routers through bgp mesh with other plops and the cause other props but within this pop here too and reflectors all the other routers are clients of these right reflectors and that make sure that the ibgp will actually scale so that's pretty much the same thing now the point of presence design for the type of modules that you have lose the customers these groupies I don't notice the water dollar something you still have this imparts the world flew by with no revenue you probably have some dedicated access writer in fact these days the bandwidth is so small people don't really worry about this these line customer connections with Cuba looking at like two megabit type speeds they slightly ball back and cost for this for revenue you need to be probably a little bit better with how you connect some of these organizations broadband now this can be over DSL what was a couple of four lines can be over the cable network okay TV could be wireless or whether it's WiMAX Wi-Fi like deploy microwave whatever doesn't matter these are broadband they tend to be higher bandwidth again they don't pay very much money for this type of access you have large numbers of them so you need to think how you're going to connect these into your backbone if they don't match you Ethernet this means you're running fibre or Ethernet to buildings some parts of one of these ISP puts a roger in the basement of a building and then runs Ethernet every single apartment so there's a very very big banquets involved ISPs are very fast internet they offer television IPTV this type of thing over that type of connectivity so we're looking like Gigabit Ethernet or even higher with maybe 10 Meg or 100 megabits per second into each apartment so the demands of these types of customers can be very very baby Roger present school we've got two dedicated Rogers as I mentioned high speed interconnect between the typically it's the biggest bandwidth connecting the two riders because of the transit link through your web presence make sure the core Rogers only have backward wings so the cool riders tend not to have lots of slots they tend to be smaller four or five slots to carry the interface cards to connect to other parts of the parks presence but the knot did not be routers are fast routers but are not physically that big the main idea is not to touch them but you should need to upgrade them you should need to change software change configuration that garages should be very stable we typically in core routers ISPs I access and uptime of about a year that's a sort of length of time between doing any maintenance work on them you should not need to touch them the bottom network dedicated model router to other ISPs so this is the ISPs front door the border router is actually very very important and protects your network from the internet it protects the internet from your network is where all your peering is done your BGP policy is done you're connected to other ISPs is done you may want to do web caching at the edge of your network as well you may want to do it in the core of the network also to fall arrivals in the back is the minimum that you need to give your redundancy so make sure that you always happen to hit the network for the ice be services and things like DNS so DNS cache where consumers to use DNA second rate for any domain names you're boosting news is probably not relevant anymore but build one pop3 IMAP mail wheeling doing all your antivirus and dispatch scanning of customer mail that type of thing so the mail services are very important www there's the server so your company web server you made me want to run web proxies as well web caches all these things are good to offer the customers because it's team keeps keeps web traffic off your international connections and makes the jungle experience faster for everybody if you're going to do most of services data centers so things like virtual hosting virtual web information content services you've got business who want to do electronic commerce and so forth keep it separate from this don't let these servers sit on this network you don't know what these people are doing they may be a compromised web server getting into your ISP services is very bad for you so you keep them separate generally if you do web hosting you have one VLAN typical design today huh so you have the router bigger than the interface connecting to an Ethernet switch each Ethernet port is a lot different VLAN you're aggregated into the router and that way you secure each customer from each other and from the internet in general don't just have one flat line with every single web server sitting on one VLAN power or Stetzer you can have up to four types of VLANs so there's quite a lot of scope network operation center if you've got multiple sites is probably worth considering primary and backup locations this is where you do your network monitoring your network management your your statistics gathering your long gathering where you run things like mrtg NFC and F done or armor speed flow you do your security monitoring intrusion detection remote trigger black hole filtering all these things all is run from the dock it's actually they important part of the ISP even a small is be set aside a small room we have a future two strings that are available for immediate access to the network you need direct access to the backbone but it needs to be secure as well we don't want people to be able to break into your knife you also need an argent band management Network which is gives you out of bag access or access to the console ports of the rioters switches on a service so I'll show you how to do that later low speed access typically well this is PSTN but here again you can see the sort of design we have a connection to the core routers we've got the access network gateway routers here so we have routers that post a network this is the vault module part you have access servers control an authentication here you've got DNS caches so the DNS resolver can have web proxies web caches we can do the service selection gateways I was controlled engine all these things can all be done this part of the network okay so giving you and well this is all you consumer say that analog access like this broadband access will look very very similar we'll get to it later but that's the sort of design you want try and keep regular small traffic off the backbone so that's why web caches local identification zone is very important medium speed access so this is your business consumer business customer links now highway you connect them they're all pretty similar type customers I same with high speed you probably put them all into one box but typically again you've got these routers the aggregation writers within the network so the broadband access module could look something like this you know it could be the telephone network connect you to the DSLAM IP so PPP over ethernet connecting to your freebie runs which is just the broadband remote access router so P Rises typically abnormal routers terminating PPP over ethernet or PPP over ATM connections on here connecting into the core again got web caches with an authentication DNS resolver for that type of thing cable system again you've got the cable access server here again connecting you to this small part of the top and then you have the access routers they're connecting you into the core some ISPs call this distribution layer if you have the router city in here controls the access between the backbone and the actual broadband module again the idea is to try and give as much accessibility of content access to this part of the network trying to isolate it from the actual backbone infrastructure the garages with these access gateway routers or the distribution routers you can do Roger these great limiting so you can set limits of what the broadband customers are actually capable of doing on the backbone it's a nice easy way of doing it on the router ISP services something like this you've got the Ethernet LAN here service network gateway Rogers again connecting into the core these routers are very important for protecting your servers from the network for example this is a DNS server then port 53 is public all that should be getting through from outside world and secure shell so you use the filtering capability that packet filtering capability of those routers to protect your services latter same for pop3 mail relay mail really is where all your customers and I bribed email so I plan even goes to your mail relays and then right to the world customers come to your mail server here to get email usually do it DNS cache is probably useful to have in the network as well it's good I need to have several DNS cache is scattered around but they also put the web cache here also make sure the filters here are very very strong filters so that providers of customers can only get access to the services they intend to get the most observances module slightly different here to different customers coming in as I was saying the games are all in VLAN so each cell phone has its own VLAN on the Ethernet switch the VLANs are aggregated on the Rogers and that way scure the access for each customer again only gives the customers Bob so if they're going to run a website they get port 80 give them secure shell access don't let them turn them to use other insecure ways but Microsoft remote access all these things are totally insecure the only way people should be accessing these servers is through secure shell ssh border module so again the border routers connected to different ISPs connecting to local exchange part make sure rogers go to fall right no fool BGP table all they carry the local routes on here so the bottom module could look like that network operation center appointed to the one router here connecting to the two cores they'd have their primary DNS that's hidden from the world view that's a very important system it's where your domain name and all your posted domain names are checked your public secondary name servers are the ones that the world sees as providing your DNS all the models from your Rogers capture them here Takei server for secure access so user authentication access to the Rogers you don't store usernames and passwords on the routers we use an external server or servers we're off to do that NetFlow I mentioned this battle and have sand and a dump so your network analyzer goes here also or if you buy a commercial - either by management Network if you're not using the same building as the data center you didn't have our demand management natural connecting to that router then we've got the firewall here this firewall is connecting the network operation center to the corporate line we quite often the knot will need to look up the customer database customer database of course is in side the corporate network you don't want your billing system and everything to sit on the dock because if somebody breaks into the dock then it can access to all your billing so you really want to keep the billing system safe and there so the NOC usually has a firewall connection to your corporate lab but this that's how we do it the I don't like network so we have a terminal server here we can make the roger consoles to the terminal server that connects into the dark of the ice cream backbone again NetFlow naval browsers you can if you get in it's the same data center you can send all the flow data to the flow information collectors so if using AF dumb and upset this is the sort of design you can do you don't have to run all the flow data export over your public back book you can run it on an out of and network so looking at this backward design rajat backbone is what's very very very commonalities switch backward is obsolete ATM frame relay has pretty much disappeared it's pretty much disappeared by so project backbone is what we have so routers other infrastructure connecting by a variety of media types can be traditional TDM we can go into optical connections stm-1 STM for we starting to see more people using Ethernet so geeky dang diggy or c7 six to eight even 100 Gigabit Ethernet is now reasonably readily available for eyes piece to connect try to standardize the pub design so if you go to multiple hops in your network designer to look the same because that way it's easier for them to scale it's easier for you to understand the network design I use be central services are distributed around the backbone that's important to me to go home pop you can't do it but as soon as you have more than one top distribute the services around there's a one breaks you have an a-cup consider having network operation center and on alternative so again if they're not disappears because the public disappears or you lose power the data center at least you can still run the network from somewhere else again most ISPs will do something like this and of course make sure you have redundant backbone links as well that's very important so not for pop3 path backward something like this you've got three parts of presents to Caracas in each I esprit services here down there custom connections of all three external connections in part 1 and part 2 you've got the backup operation center for the primary one you get the idea try and distribute services around the network so the backbone links ATM free really has virtually disappeared huge overhead extra equipment and share with other customers of the telco this is not good it was never really very good offering because telcos oversold what they had MPLS has pretty much replaced ATM and frame really as the telco favorite so the telcos have adopted MPLS really as a replacement for atm the other way of doing it is the least lack of general infrastructure circuit this is most popular with the backbone providers my view of objects Metro Ethernet this type of thing people are using Ethernet de Ethernet and so forth even for medium distances some places I see either gigabit ethernet links 100 200 kilometres long not been used for ISP backwards because it's the cheapest way of providing the infrastructure now the long range backbone links can cost more it's important to plan for the future and this means timing at least two years ahead but you need to stay in budget try to stay reasonably realistic we are trying to do emergency upgrades because you've run out of capacity can be hugely disruptive to the network so you fought so far long distance you need to be very ambitious try and fight as far ahead as you a low sufficient capacity alternative paths sufficient depends of business strategy some ISPs leave as little as twenty percent overhead this is really very silly because if a link goes down that backward becomes congested the whole network becomes congested so this is not a good idea typically leaving sufficient free means having 50 percent of the external capacity available which means that if one external link was done the other one can take the full load without any loss so I generally find that 50 or 60 percent free capacity is a design guide from many iostream animals some eyes please push it to 20 percent you can tell by the price of the services if they are cheaper then they have less overhead available in the network so if the network breaks you're more likely to get serious congestion some businesses actually choose 0% overhead they deliberately run the network congested and instead they tried by rate limiters to limit what the customers can do this is very very short-sighted it doesn't scale or grow the internet and it's just one of the worst options available but there are people who do this attempt to offer very very cheap services and that only way they can do it is for having zero redundancy in the network the 50% is really ideal so a long-distance links make sure you have the redundancy cause as big capacity as you can within the backbone but with in the metro area so within the city it tends to be easier if you've got more than one pot because it's easy to get local fibre or you can lay it yourself or you can use other types of technologies to link the backbone together so generally all the cities around the world it's very easy to get big capacity locally so think they do things like this you know have duplicate links between the different parts of presents and that gives you a very good solid infrastructure network okay upstream connectivity and peering we've seen this already I've talked about this was it yesterday transit provider gives you access to the other networks could be local regional more usually the whole internet transit providers need to be chosen wisely one doesn't give you redundancy too many too difficult the load balance and as I said yesterday at least to no more than three so the common mistakes I mentioned yesterday too many transit providers or is get transit providers but you don't have diversity because the transit providers use the same satellite or the same submarine cable or the same Overland cable and so on so choose transit providers that a different infrastructure in a toddler networks Peter as much as you can it's very important to go and get peering everywhere clear with more local ISPs obviously don't peel with your customers peer with your local I speeds of the other ISPs in the region would knock your customers peer with them it's much cheaper and paying your transit provider if there's an internet exchange point be at it internet exchange point doesn't cost anything all it cost you is Ethernet port - you play at it you don't pay for traffic you don't pay for transit so peering of budget account is very important it's how the Internet has been built by peering I mentioned the common mistakes some transit providers talk about exchange points when in fact is not an exchange point it's a transient business recharge for traffic if you charge for traffic I can exchange it is a transit business it is not an internet exchange party also another mistake not working hard to get period you need to peer as much as you can I mention yesterday is please book special the Middle East will go to Europe get my transit in London or Frankfurt and ignore the exchange parts that are there and they say although you know we don't be happy to pay for all the transit but if they went to the exchange point they get 50 percent of the traffic by Piron transit may be cheaper the peering but this is very rare peering is usually cost you nothing more than the ethernet port you connect engine roof other mistake isn't ignoring or avoiding predators because they are competition they are valuable clearing partner because it makes the local Internet experience better for everybody and it helps grow the local internet economy so private interconnection - is basic region to connect the networks with cover this and so the private interconnection can look like that you've got your peering routers we run I mean to be here around ibgp there this peering round you must know default right doesn't have the full BGP table it's all they got the domestic prefixes inside the essence and this peering Roger is used for all private interconnects so the public internet the weather service provider goes to an exchange point strange prefixes it originates into the writing system my SP will choose who to deal with you don't have to deal with everybody at an exchange party bilateral peering it's like doing private to connect multilateral peering if the exchange point provides a route server then they provide all the prefixes of everybody who uses the right server so those that you that you get so it's something like lasik of the iris we have to go over there then peering budget is at the exchange point and it peers with other ISPs that they it runs ibgp with the rest of the backboard and ebgp with i speakers that has no default no full BGP table only on the domestic prefixes I had my IV GP so that you need to be careful configuring this router so more from the domestic backbone don't originate prefixes as well as not the default under full BGP table need to filter in and out don't forget your filter if you have a second way to exchange for doing something like this but a second router in second circuit going for a different place in your back work that way you get done in situ exchange by and both these routers is left clear with the ISPs at the exchange now for the upstream transit connection so two scenarios transit providers in the locality which we expand with this chief so get big capacity transit providers a long distance away undersea cable satellite or more cross country flag all these states the different scenarios if you've got a local transit provider connect something like this you're born around here connects to your local transit provider this may be internet but a little local Wireless updates but it could be just a cross across time across the city so it's a short distance you connect to your trans providers access router and they give you onward transit to the rest of the intimate as you get the default route or the Foo BGP table you do your policies as I discussed before and you do your packet filtering of the border router as you need if your transit provider is a long way away in another country another continent on the side of the world then you need to do this we need to take your ball around her and move it close to your transit provider this link is very expensive and you need to be able to control what runs on that link so typically you go to the trans provider by the vehicle or center or find the data center that's close by of where they are at put your router in there and they get a local connection to the transit provider to run ibgp with the rest of us v1 backbone your EB GP with the transit provider they are what they are two riders this is the best thing to do put your border router next to your transit providers backwards so this is if they're a long way away this is a long-haul circuit as I said another country another continent for example you're here if you're connecting to an isp and hong kong put garage in hong kong because then this roger can control what goes in that link if you do Revlimid package filtering all the rest and you get best use of that particular connection this is really really strongly encouraged many if not most of the bigger ISPs I know we'll put a router next to the transit provider if that transit providers are long distance away these long-haul circuits are very expensive very expensive if your transfer provider is local you can change the capacity anytime you can very easily it's just like a local Ethernet connection but long-haul you've got latency possibility of congestion buffering issues and all the rest the upstream provider uses just a standard access writer so if you plug in a long-haul link you putting a lot of pressure on their access router to give you enough buffer space so this moves the buffering proper away from the transit provider you do tune your router and you can make sure you get the best performance of your link the other thing is if you're in a remote data center and you don't like your transit provider you can sign up another one and then disconnect the old one so it becomes very very easy to migrate connections between transit providers with no downtime did you want to change in transit provider and you just plug into that router you have to arrange to move your long-haul circuit into another data center and that will not happen without you losing any packets that can take days to happen and that will cause packages on the natural so being a remote Colo gives you huge flexibility for running your network this is why many ISPs will put a router in the remote color if your transit provider is distant other paisa consider following each remote hand support so if you're going to put a router somewhere overseas you can't go into an airplane and fly there if there's a problem so you need to find some D in the data center and then each data center usually has something who can plug or unplug cables power cycle equipment if you need to upgrade the router you can send them a new one and they will swap it you need people to do things like that to get arrangement with what's called robot ads and they can do this basic work for you you also need to support contracting equipment vendor you can't just get equipment there with no support the writer breaks you don't have to ship it back you to do something but getting it fixed and that cost money so get a support contract an equipment vendor so that so they can fix issues with the router or whatever locally it's actually sensible to consider two routers and to long four wings to do something like this so that the transfer provider have two routers and two legs there so rather getting upgrading one leg to twice the capacity get another length on the same capacity and then you can load balance on the two of two independent circuits and even consider connecting two exchange party guests you get the transit provider but most data centers have an IXP as well so you can connect to the exchange point also and get free period so summary of this primary connect simple private peering we've looked at the design needed from public interconnects put the roger at the color at the exchange part local transit provider simple upstream connection long-distance transfer of either put the router in the road cool overboard data center and connect to the transit provider there and also consider going to an exchange point there now the IP addressing we've seen this all perform so I've talked about this already don't use private dress for dressing the backbone use public addressing otherwise it just makes it really hard to run your network the well-known problems I've listed all the problems of trying to use private addresses for the backbone it's really a bad idea so don't do it you want to do it make sure you have a net on the edge of your network so you can do the translation so what else infrastructure security is not approve for giving private addressing you can still be attacked inside or by your customers of a reflection techniques it's very easy to attack through an app if it's troubleshooting harder because you have no internet view from the routers other ISPs cannot distinguish between done and broken path MTU discovery and be brought in as well so the summary always use global writable IP addressing for your that's very important for high speed so do that address planning I mentioned this already address block to the right of loot bags address block for the infrastructure part of grumblings silver policies for v6 we've covered all these already customer assigned address race according to needs for v4 v6 to get a 48 or a 56 or 60 or 64 or whatever but the customers into ibgp not boss PF so i looked at the address plans we've done this before in the labs phase to wow these things you're likely to get an adjacent one before it's all gone so this doesn't work anymore what was the difference you need to doctor so registries will usually allocate the next block to be contiguous that now listen this is not going to happen for me for anymore v6 it still will do documentation is quite important so documented all your address allocations i talked about the different tools for documenting addresses the other day whether it's a flat text file or spreadsheet or one of these online tools that you can use or use the writing registry is entirely up to you that please document the address allocations for writing protocols we have igv current infrastructure addresses 20 point lengths so we have 11 eius eius but EGP we've got exterior gateway protocols of bgp version 4 we use is is to scale the backlog all the point-to-point links let the righteous talk to each other and we need BGP so that we can scale to the Internet and give us a policy between different autonomous systems so that's why we have BGP as I showed you of Monday comparing them to automatic neighbor discovery for eius eius specifically configure the peers for BGP you can trust your IP routers but here you're connecting outside the network prefixes go to all my GP writers whereas BGP we set administrative boundaries so by esaias carries infrastructure addresses only you keep is is as small as possible for efficiency and scalability of network with beach people carriage a customer prefixes the internet prefixes and of course the EGP is an independent of high-speed network topology hierarchy of the writing protocols one spear higher size is the IGP we talk BGP to other ISPs which are BGP at exchange points we talk static or BGP to our customers because each of the customers needs the Internet we need to run BGP of top of is is in our backlog so that's why we have Maya sighs and then I BGP sitting up top okay so choosing an IGP I've given you slides compared for speed by esaias think one that works for you so pick one of works for you I want with the recommend we didn't eius eius in the workshop as you see it's actually quite easy it's quite easy it's more secure than course yet because it runs on the infrastructure OSPF runs on top of IP so you can attack voice fear using IP and the other thing with ipv6 you need to run two separate OSPF one for us we have our D before and the other voice Claire for ipv6 whereas bioscience you just run one protocol to topologies so I assign is a lot more sensible for me anyway then user or SPF keep the routing table as small as possible if you can count the Rogers and the parked part makes the backyard goes on the ITP entries you should have so even for sizable networks your is is will be nice and small so Roger loot bags backbone mine plan to point links and network addresses of any lines having an IG p running on them use intermodules education even though i Esaias runs on the infrastructure as you saw in the last it was quite easy to set up identification so may as well use it it stops accidents from happening using SPF use authentication as well summarization if you can that will help reduce prefixes at ISI more recommendations use IPM number than customer point-to-point links if you meet a customer find apart link put it into BGP instead don't carry it in is is because that way you can keep ISI are small BGP in carrying bonds of grief it's trying use contiguous addresses for the backbone bind and then summarize for every we can don't summarize routes or do pack your dresses because ibgp needs those for the next hop and fiber use I new GP for anything which doesn't contribute to I GP so things like internet routing table should be an IV GP customer addresses customer point-to-point links and you drive going down up whatever pools none of those need to be in your I GP but BGP use neighbor authentication showed you how to do that use peer groups it speeds our configuration efficiency less chance of making errors and so on communities I haven't had time to show you unfortunately communities make doing policy much easier use the Roger fain to hierarchy told you about that show you how to design into this copies yous right reflectors that means your PGP will scale for security while we look at the infrastructure security the network security I have to find out the security is not optional as a service provider network operator you have to protect yourself you're the target you have to protect your customers from the Internet your customers assume that you know more about security that they do you have to protect the internet from their customers as well you don't know what your customers are up to they could do something that is bad for the internet and the thing is if something like that happens you might end up being the receiving end from other people I've only got general recommendations here we could spend a week during a security workshop so but typically for router security usernames passwords don't just have admin node or something like that is the access every single person who accesses the router give them a username because then you know exactly who's logging in you have a proper audit trail of who's logged in if you just have a generic admin account you've no idea who uses it no idea who is the fan who's got the password so make sure you have usernames and passwords of every single user put filters on to the access the vty pods when you tell them to SSH to the router put a filter there that limits who can SSH gently what high speeds do is they have trusted posts in the NOC and it's only those trusted posts and the backward ranchers they can get to each other you don't want the public Internet to be able to get any response from your router so blocking access the rest of the world very important also use tax tax is a server that runs on a UNIX system that hands out username Bob hazard passwords when people log into route if you log into the router it asks the target server for what the password is it's a secure link between the router and learn so as it doesn't ask for the password it compares strings of the password so the hash of the password you entered versus the hash of the password stored on the server okay so centralized authentication it means that if you need to add a user you add it to tack acts over and you need you to get access to the writer if you need to remove a user your multi matically it just remove them from the tack server database and they no longer have access you suddenly leaves your company you can simply remove from them the tantric server and they no longer get any access if you store the enable password on the tac-x server you want to change it you change it from the server and then it's automatically changed of all the routers everywhere else so it's a very good way of quickly making changes or rather security don't use talent you secure shell and make sure you secure shell as a good strong key you need to use 2048 bit or higher modulus otherwise the smaller ones can be quite easily attacked so you secure shell don't use tablet vty filter should only buy the block access don't allow any access from outside your network or from your customers denial of service attacks and need to be able to deal with those so the best way of denying who knock on door is still there I think it is dealing with things like remote trigger Blackwell filtering so if a customer of yours is attacked or your or you want to block access the particular site for some reason you need to install a reward trigger blackballed set up on your backdrop have a look at the team covering website to find out how to do that you can find links starting off from here make sure you get effective filtering of the border writers that quite a lot of things happening that you should not allow into your network things like Microsoft net buyers you see a lot of that that should be running on the public Internet the lots of are viruses worms dos attacks on routers and so on testing various ports for vulnerabilities usually on Microsoft Windows you need to block those at the edge of your network very important to do that study customer connection I've talked about mini cast RPF on Tuesday wasn't it make sure all your static customers have RPF on interfaces and that way you stop them sending packets apart from those you are expecting network operations center make sure is protected as well generally what high speeds do have SSH access into a Bastian server and then the NOC and from that server you can access the routers and everything else so when you're traveling you need to log in to your network to do maintenance log into that one machine all that runs is SSH and then you can go from that machine to others in the network make sure the corporate network is behind a proper firewall that's quite important as well as I said your customers should not send any IP packets ITIL internet the source address other than the dress you have allocated to them that's what the RPF check on the interfaces will actually do now I took my management this is important as well out of my management is not optional this gives you access to network equipment and types of failure for example here in the lab you have been accessing the writers out-of-band you haven't been talented to the routers you have been talented to a thick access server the gives you access to the console of the writer that's what I mean the edge of that management you're not going across the network you're going into the router console but you're doing it from the comfort of your office or home or wherever you are this ensures quality of service summers minimizes downtime minimizes the repair time and eases diagnostics and debugging so a typical example you can access server put a bottom on turn to elect yourself darlin if it's a remote site quite useful to do that the cops'll boards of all network equipment equipment is connected to the serial ports as you take all the consoles of the routers plug them into the XS er you then turn that to the access server and it will map you through to the console port and this gives you full artifact access so it looks like this got the routers here you can all the console ports connecting to an access server you can then tell it or SSH access server maps you through to a console port that you can then use that so gives you console access to your routers without going into the data center so that means you stay warm you can stay in your office and you can manage all the equipment if you lose IP access if it's a remote site you can probably put a motor on to the access server so you can dial in to the water and then get access to the network that way if you break the whole connection you can still dial in and fix things statistics garlic also happens in our Japan network so routers using NetFlow syslog is enabled the thing is the management dangers all set by UDP so it can be congestion and sensitive so this ensures that management data is not lost at the time the network is being congested so this gives you maximum information under all circumstances so there watch your statistics gathering done the syslog system logs net flow information all that kind of stuff you want to capture the network final piece the test lab it's very important to test things it's not a good idea to try out services on the live network so you want to build something that looks like a punch presence keeps approaches to the side so you can try out new services or new configurations actually with the dynamics that we're using here for the workshop this is actually very powerful tool to let you simulate or test your network infrastructure configurations so more ISPs and now using dynamics to test new configurations and unless you find and fix potential problems before they go into the network so how do we do it some people will Danny to it equipment to the lab other ISPs the forwards buying Rogers so they're just by Rogers in advance three months earlier than they need them and you can put those in the lab for testing other ISPs use lab equipment as hot spares so rather than buying a 24 by 7 brick fixed contract from the vendor they just buy and break things so things break you send it back to factory and get it fixed they buy extra equipment and they leave in the lab if something breaks the network we take clip entitled lab it goes into the network and descend other part back for repair the vendors don't like you using this because they can't sell expensive maintenance contracts but expensive maintenance contract won't fix your network faster than you if you can take the hardware part of your lab into the network we usually needs walking from one room in the data center to another room in the data center you can fix it in a few minutes your vendor probably has to drive from their office they're going to come and look and see what's wrong even if they say four hours break fix that's not enough right it's too long if your network is broken it's much quicker to go next door to the test lab put on the line card put it into the Roger that's broken and that it comes probably having no more than two or three minutes damn time we did that in unit we brand our lab was full of hot spares we just had a break fixed contract we did not have a 24 by 7 Cisco didn't like us for left but too bad they could not give us a 5-minute guaranteed fixed time of something if Hardware group can't afford a test lab set aside one spare outlet server to cloud new services I should add it here used items use dynamics as it's running a listing which you can prior configurations forth before you even put it onto the metro but never try out anything you on a live network because things will seriously great operational considerations why I design the world's best network when you haven't thought about good operational practices so the first one maintenance never work a live network no matter how trivial the change they see establish maintenance periods that your customers know heart to see four to seven Thursday 4:00 to 7:00 is very typically used in the industry Tuesday 4:00 to 7:00 a.m. means you spend Monday getting ready Thursday 4:00 to 7:00 means you can spend Wednesday getting ready where as we try and do Monday 4 to 7 you work the weekend you do Friday 4 to 7 you probably have to work the weekend fixing things right so maintenance never happen happens on Fridays or Monday Tuesday Thursday is very very common there SP networks differentiate between customer support and the network operation center the network operation center runs the network customer support supports your customers different people customer support fixes customer problems knock deals with the backbone and Internet if customers complaining internet problems they thought the customer support customer support passes onto network operations different thing network engineering other people who design the actual infrastructure then during the next generation that trialing new services you writing design new things they should not be doing customer support right network engineers do not talk to customers customers talk to customer support that's all even network operations does not talk to customers network operations talks to other ISPs and the customer support you need to keep this separation between the three parts of the organization even after three different people if it's a small organization three people it has to be very clear the customers speak to customer support network operation center deals with the running network peers transits network engineering designs the next generation communications with the NOC you need to know the contact details for Knox in your upstream provider and so your network operation center has to have very good relations with your upstream the daughter Putra lations with the upstream you cannot fix anything if there is a there's a telephone system for sobriety and 12 the line hug DBA you can use that some people are on using that one only natural copper centers can use that system so at least if you make phone calls of math you know something's going to answer and it's good going to be an avid engineer design summary of this keep it simple and stupid so that's what to remember keep the network design simple keep the network design stupid don't build a complex network it becomes complex it becomes very very difficult to rock redundancy is important security is important also make good use of the technology that's available make it easier for yourself above all ensure quality of service for your customers that's the most important thing that's really all I have to say ISP design so it goes over some important things mean some probably imply nice to you than they do to to other folks it all depends on the size of your network but it is important to pay attention to how you design your network the best way of of scaling it because if you start with just a random-ass things connected to each other it just gets very very complicated to try and and wrap and so what to try and keep things as simple as possible the presentation is based on what I've seen a lot of ISPs do it it's not made made up and again any ISP I help I've worked with over the years I push them in this general direction so if you get a nice big scalable simple design for the network

Channel: APNIC Training

Views: 61,403

Rating: 4.9320111 out of 5

Keywords:

Id: GBO7739TfJE

Length: 69min 20sec (4160 seconds)

Published: Tue Mar 05 2013