Data Center:Network:Cisco:Nexus:Advanced Virtual Port Channel (VPC) Designs

[Music] welcome back everybody to INE CCA data center and nexus switching class today we're going to continue our discussion of the virtual port channels now that we started yesterday we're also going to be talking about fabric path which is an alternative basic layer to basically a layer 2 routing protocol that we can use in the inside the data center now before we get started with the additional topics and continue with VPC if anyone has any questions about what we talked about yesterday feel free to type those in now a couple questions I had here to start one was if I had about the peer keepalive on the Nexus 5 thousands so as I was mentioning Nexus 5k is mainly a layer 2 only switching platform it does have the layer 3 adding card that you can add on some of the platforms not all of them take it but that would allow you to do like OSPF routing or erp routing basically make it like your your layer 3 default gateway or your end hosts but normally by default that's going to be mainly a layer 2 only platform so the question then becomes if we're doing VPC the virtual poor channels and we need that layer 3 keep a live link the VPC keep a live length appear keep alive then how do we do that if it's a layer 2 only a platform so there you'll see that there are actually some basic layer 3 functionalities built in even if you don't have the add-in card so you can configure interface VLANs you can put IP addresses on those and then you can at least send basic pings back and forth between the devices so if you don't want to use the management 0 interface you could take in band data links like to physical 10 giggy's you could channel those together put them in an access VLAN and then use the switch virtual interface the layer 3 interface in order to be the source and destination of the to keep alive now we really the design advantage of doing that is that potentially if your management link goes down and then the peer keepalive is still there but again when we're looking at the design difference between the 5 K and the 7 K 5 K really isn't a redundant for him to begin with because there's no additional supervisors there's no additional line cards it's basically just the fixed switch either the one you were the two you depending on the platform that were they're talking about so you could use the SV eyes that would be another configuration option there there's a another question would the commit rollback help when the VPC configuration was messed up in a V BC change you can there there are some ways to do the configuration management with checkpointing it's it's not as it's not as intuitive as it is in like iOS XR where every time you make a change or making a commit and then you can clearly roll back and forth between the between the versions I'm not going to show that now but after class some of the additional videos that we go over just some more than management techniques in detail of nx-os we I will be talking about that so when we go over examples of you know sending up SNMP and then actually looking at how do you send traffic to the collection station setting up NetFlow actually sending it to a net flow collector so there will be some some more targeted videos on those particular topics if you want to see exactly how it's working from like a a real design point of view now one of the issues that I ran into towards the end of class yesterday was with the the Nexus 5000 we were looking at the active active fabric extent or the FEC's topology where basically we had the two parent switches the Nexus 5 case then they were going down both dual homes to the fabric extenders in a V PC environment now one of the configurations I was showing is the switch profiles or the config sink that would be used to make sure that the configuration stays the same between the parent switches now the the first portion of the configuration I was doing the initial commit worked the second committed committed had an error message said that the verification is is was failing and one thing I forgot to mention about that and I want to bring up the documentation here is mainly if you are using config sync you basically have to use it exclusively so you pretty much use can fixing for a hundred percent of your configs or they're not going to be able to use it at all so from the main documentation page we go to products then to switches then to datacenter switches then to the nexus 5k the configuration guides then this is going to be under configuration fundamentals or system assistant management here the assistant management and then switch profiles so like I mentioned earlier you definitely want to spend some time reading through the documentation because there's a lot of specific caveats for the individual versions and then the different hardware platforms that we're working with here but one of the issues here if we look at the the configuration validation there's a check that's run that's known as the mutual exclusion check or the mutex it says as part of the commit process the mutex secure occurs on both switches so your local peer and the remote here basically to make sure that the one you apply the config it's actually going to work okay but the this sub node here is what the problem that I specifically ran into you it says prior to release five one three and we're actually running five one three so this this applies to the version that I was demoing it says an interface config could be partially present in a switch profile and parsley present in the running config as long as there were no conflicts so that was in the older versions starting in release five one three port channel interfaces must be configured fully either in the switch profile mode or global config mode so basically what this means is that since I had configuration going on before on the 5k and we were doing the the port channel config we were doing the V PC config then we had some access VLAN configurations on the other switch ports when I went to apply the config sync it basically said well you can't do the pour Channel in config in the switch profile because you already had it outside in regular global config so you would basically have two options what you can do is well you could just not run config sync at all and then just manually make the changes but what you can do if you do want to use this feature and you didn't use it from the get-go there's an option that allows you to import your previous configs so you say import the running config under the switch profile and it basically then makes those commands be under the control of the config sync process as opposed to the normal low Balcon Fager so had we done it using config sync from the initialization so we boot up the switch give it a management address give it you know hostname configure cisco fabric services over IP you configure the switch profile then from that point forward you can use config sync but if you didn't do it in the correct order to begin with then you're gonna run into those to those failures so I'm not gonna go back and go through the config again uu I definitely will want to try this out within the scope of the your CCI preparation before you actually get to the lab but basically if we were to delete all the configs and then start it again in the in the same order then it would be fine okay but again a lot of these features in nx-os are very highly subject to their order of operations where you have two identical configurations one works one doesn't work simply based on the order that the commands were entered in and really the only way to figure that out is either to try it on the command line or to read through the documentation and see the other particular release notes for that individual version okay so let's spend a couple minutes reviewing recapping real quick what we talked about yesterday some of the different VPC topologies the first one we started with here was the two Nexus 5000 s and then we had the the downstream neighbor which in this case was a router router one and the the key portions that are making up this topology again the first thing that we have is the peer keep a live link here the peer keep a live link in this particular case was running on the management 0 interfaces of the 5ks we also saw some examples on the Nexus 7 K that we were running that we were running that in band on the data plane so here these were the management 0 links that's gonna allow us to establish basic IP reach ability between the neighbors once we have that up then we have the peer link ok the peer link again was what we use with the cisco fabric services over Ethernet or CFS oae in order to actually synchronize the control plane between the devices so not only the control plane in terms of like the MAC address forwarding table the ARP cache the igmp snooping database in the case that we're running layer 2 multicast but also that's going to be used to exchange the consistency parameters which is going to be things like the speed the duplex the MTU any of the particulars of the VPC members we need to make sure that those are synchronized between the V PC peers otherwise the the member interface is going to fail when we actually try to to allocate ok so once we have the keepalive done once we have the peer link done then the next step is to actually configure the V PC member ports so in this case this was the 2 Ethernet 1/24 x' we put these in an interface port channel like interface port channel 1 whatever local assume you if again number we say that this is a V PC so we'll say V PC 1 and then whatever our layer 2 config is like switch port mode trunk normally we would want to say like this is the spanning tree port type network depending on what type of downstream device we're going to in in this case if we're going to an endo Sui would probably say the spanning tree port type is edge so we wouldn't be running the normal bpdu exchange with that link would not be part of the synchronization process of rapid spanning tree protocol but again it's mainly these these three main components what we need to deal with they keep alive the peer link and then the actual member ports ok the second example we had then was doing this from the 7 KS down to the 5 case as we saw these these 2 7 thousands they were VDCs that were part of the same physical chassis so the additional caveat of this config is that on the management 0 ports they're not able to talk to each other because that's the same physical link that's actually used on the supervisor so if these were two separate physical chassis x' we could run the pure keepalive link over management but in this case since their VDC is in the same chassis that's not going to work so what we did then instead was to take some of the M ports so these are the layer three m1 module ports we put those into a port channel then on the pour channel whatever interface port channel one two three we said this was not a switch port so that was making it a native layer 3 routing interface and when we do this the other design consideration is to make sure that it's nav RF so we said it was a v RF the member v RF whatever the name was I said it was pure keepalive and this is just to make sure that it's isolated from the rest of the the data playing traffic so if you have some sort of actual like looping in the network because there's a you know maybe a cabling problem or some control plane issue then the peer keepalive link should not be affected by that because if it's in a different virtual routing of forwarding and since a different v RF then traffic cannot move from the global routing table or the default v RF into the the user-defined v RF which I called pure keepalive in this case ok then likewise once the other peer keepalive was up we have the peer link okay the peer link we want to make sure that this is at a minimum to ten gig e links that are in a layer two trunked port channel so it's a twenty gig port channel once the cisco fabric service is over Ethernet has synchronized between the two of them we would see one of them is elected the primary role one of them is elected the secondary role for the V PC peers then we can go on to our final step which is to create the V PC so in this particular design we had two separate V pcs we have V PC 51 or whatever the number was that was going down to the first 5k which exists on this link and this link then the second V PC 52 that's going down to the second 5k on this link and this link so this is a real important point we need to make sure that we're actually assigning the V PC member ports to the correct physical links one of the potential common config errors here would be to say that both of these are 52 or that both of these ports are 51 when in that case that's not gonna work it has to be going down to the same downstream switch now we're also going to see another more enhanced example of this today on a design that we know we call back-to-back VP sees where the seven k's are doing a vp c downstream towards the five K's the five K's are also doing a vp c upstream towards the seven KS and we have four physical switches or in this case three physical switches because two of them are made up of DVD C's but normally for physical switches that logically look like two switches so a pair of them upstream a pair of them downstream the pair upstream looks like one Bridge the pair downstream looks like one bridge and then we we run in we have the case where we have the the sixteen or the 32 way 10 gig port channel so 160 gig or 320 gig depending on the particular models and line cards that we're actually using for the member ports ok the next variation we had then was going down to the fabric extenders okay this is where we have the active active forwarding for the exes so the first nexus 2000 was paired with the 5k 1 and the 5k - so this means that there's going to be a V PC here and here and a second DBC here in here now again the potential problem we run into with this is that now we have two separate control and management planes which are existing on the 5 case upstream and they're managing the same downstream fabric extender so this means that if you configure in access VLAN on one of the parent switches and you don't configure it on the other parent switch then you're gonna have some problems with your actual data plane forwarding on any of your hosts like your physical servers that are actually attached to the defects is so that would be one of the reasons why you may want to use the config sync process between the five K's because we're referencing the same basically fabric extender host ports that are going down to the actual end devices but then again as we saw the caveat you run into either you have to basically use config sync for a hundred percent of your configuration otherwise you it's gonna allow you to do commands in global config and the config sink remote at the same time that overlap so you basically either don't use it or use it for 100% of the config okay the next variation then was similar to this topology with one minor change that we have an end host that is now dual home to the fabric extenders so in this case there's actual actually multiple levels of VP C's here so we have a VP C that's going from 5 K 1 to 2 K 1 from 5 K 2 to 2 K 1 a 2nd VBC that's going from 5 k 1 to 2 k 2 and the second VPC that's going from 2 k 2 down 2 or 5 k 2 down to 2 k 2 okay then there's actually a third V PC that is configured on the fabric extenders themselves that go down to the end host ok this is what is known as the enhanced V PC so it means that we're doing active active from the actual end host up to the fabric extenders and we're doing active active from the fabric extenders up to the other parent switches now this particular topology is not supported in all cases it's not supported on the Nexus 7 K when you're using the X's as attachments to the m1 line cards it's also not supported on the older versions of the Nexus 5 K so pretty much only the newest versions are gonna support that but configuration wise it's just a normal V PC config between the two five K's we configure the poor channels and then say that those are the switch port mode effects fabric those are the effects associate so those are the downstream links towards the fabric extenders then on the host ports from the effects is down to the actual final destinations we just simply configure that as a port channel on both sides we don't need the V PC keyword which is a keep keep point here but it actually ends up to be a V PC so you have active active from the host of the fabric extender and then again active actor from the fabric extenders upstream to the the v case okay there's a question here if we can you if we use config sync under profile 1 and add some configuration would we need to enter config sync instead of config T yes so in general if you are gonna use config sync you need to do it for pretty much everything otherwise if you end up in those inconsistencies where some commands were entered in a regular global config some commands where internet config sync the next time you go to config sync and try to commit a change it's going to give you that failure message because of that mutual exclusion check that mutex check and if we looked at the you could see this in texture let's go back to the the documentation there is a show command that will tell you what is the what is the failure so if you look at the show switch profile the name and then the status while you're in config sync it'll tell you if there's an error because of the mutex check the mutual exclusion so in general and unless you have already used can fixing for everything you're not really gonna be able to use it it's more like from your initial implementation you start you start using it and then you're going to stick with it okay another potential design issue you need to think about with this is that if you're using config sync you need to make sure that even the port numbers are the same between the switches so here we have the 5k s going down to the FEC's a--'s I need to make sure that the ports here which are a 1 4 & 5 are the same numbers a 1 4 & 5 that are on the other parent switch down to the facts otherwise if I try to apply a config change on either of these basically member ports of the V PC then it's not going to apply to the correct interfaces on the the other remote peer of the config sync so there's some additional design considerations you need to take into account with that ok there's another question here in the EVP see now the V PC number and the FEC's number on both the primary and secondary must be the same correct so if you're doing active active from the parent switches down to the FEC's is pretty much 99% of the config needs to be equal between them really the only thing that would be different between the five K's would be like their host names their management IP addresses and then you could have maybe like different security policies on them but generally you would want that type of stuff to be the same so anything that's really related to the data plane config like your VLANs your VP sees your trunking config all of that is going to be have to have to be pretty much 100% equal between the two of them if you want to use active active or if you're using config sync along with active active there's a question so the problem of interface numbers can occur only when you are dual hummed well the problem what the interface numbers really only occurs if you're doing config sync so you you don't necessarily have to match the numbers if you're doing the configuration manually we could we could plug that or we could wire the switches however we want it's usually for clarity that you do use the same number so it makes the management of the config a little bit easier but it's not really a strict requirement it only becomes an issue when you use config sync because if I have a config applied to e11 and I push it over to the other switch and there one one is not actually wired to the same destination then there's going to be a problem with the the end result of it there's another question here there's an option to import specific interface config only to the config sync that's right so you can say when you're importing the config and let's just search here for import importing configuration importing a switch profile you can say I want to add just that particular link to the profile so this is gonna say take the commands that was were under Ethernet 1/2 and now make those to be managed as part of the switch profile you could also say basically to to import everything or to import everything with the exception of a particular link so the idea for this is that you you started your configuration not using config sync you want to use it later so then you have to import what was in global config to the switch profile so that you don't run into that mutual exclusion check error there's a question or a comment here show run switch profiles helpful command to see what has been configured with the the config sync that's right so Ian global config you'll see under the switch profile exactly what has been changed under the profiles so basically what config sink owns versus what your your global config owns so after class once we go through some additional videos on you know more specific configuration examples of these I will have some some more detailed examples on the config sink process but usually as long as you do it in the correct order of operations you normally don't run into these problems you just need to make sure that basically everything is 100% the same not between the two platforms okay there's a question on orphan ports does this imply that when two switches are VPC peers that every end device must be VPC attached - yes it does and we're going to explore this in more detail today as well so the question is saying if I were to have two V PC peers so let's say we have a 7k 1 and a 7k - ok between these two switches we have the peer link so the peer link that's our control plane synchronization between between the the two different control planes two different management planes and then we also have the the peer keep a live link hey people like link again is going to be used for just the UDP ping back and forth between them normally what we would have then from the the V PC peers is their downstream facing links are going to the same devices so everybody is basically dual attached so if we have a switch downstream let's say that this is like a an axis which of 2960 yeah we're going downstream towards them this is gonna be one of the members of the V PC so we'll say this is V PC one this is a number mother member of the VPC V PC 1 so this downstream switch this is not considered an orphan because it's dual attached to the V PC domain this is ideally what we want ok what would be considered an orphan is if we had another switch over here let's say another 2960 that's going only to one of the V PC peers now the reason that you do not want to do this first off is that if I have a host that is coming in on let's say Nexus 7 k2 so traffic is coming in here the only way to get to this orphan switch is then going to be to go across the peering and the peer links aggregate bandwidth is generally going to be much lower than the aggregate of all of the VPC members and then possibly of the the orphan courts so we don't we ideally don't want to use the peer link in the data plane we want it mainly just for control plane synchronization if this is one reason you don't want to use the orphan ports there's also a case we'll see today when we get into different failure scenarios that if one of the switches goes down okay well obviously if 7 K one goes down here the orphan port is gonna be isolated because that's it's only physical connectivity but let's assume here that the first switch goes down so it crashes completely the primary role of the V PC is then sent over to the secondary switch so originally this was secondary this which now becomes what's known as operational primary so it's configured as the secondary based on its priority but it's actually running as the primary primary ok then at a later time the primary comes back as to this guy's okay now the issue becomes that there's certain cases where this orphan port is not going to be able to send packets into the data plane regardless even if even if the the upstream V PC P R is is working and they're kind of odd cases we will take a look at them today but to just avoid this issue completely if your switches are V PC peers you generally should not be using them for orphan port connectivity you should be only using them for V PC member connectivity for any other devices that are only single attached you should move them to other switches that are not the V PC peers so a regular switch like your kettle is 6500 that's running normal rapid spanning tree or a multiple spanning tree that's not going to have issues with the orphan ports because we're using the normal spanning tree forwarding rules but in the case of V pcs we're using that modified V P see check rule that says if a packet comes in the peer link I can't send it out the member ports of the same Navi PC member that the other remote peer has but if it comes in the peer link I can send it to my orphan ports so the forwarding rules change and they get a little bit more complicated so really just to avoid this in the design to begin with you just don't use orphan ports on the other V PC peers at all there's a question there's only if it is only an orphan port if the villains are the same as the V PC domain correct so if you're doing like layer 3 routing or if it's a different set of VLANs that are not part of the members then it's not considered on orphan so if we were to say on this 2960 that is not an orphan let's say that it's using VLAN 10 down here and VLAN 20 but the other switch that's single attached if this is using VLAN 30 for this particular VLAN it doesn't count as an orphan because those the V PC forwarding rules are only applying to the specific VLANs that are part of the V PC members okay but the issue then becomes further than this on your peer link you would want to remove in the trunking allowed list any VLAN that is not part of the V PC member ports so in a normal design you would not be allowing VLAN 30 to go across the pure link which means that if this is the only layer to interconnect between the V PC peers VLAN 30 is going to be isolated to begin with okay so you could add another layer to trunk link that's running normal spanning tree and then use that to forward the VLAN 30 traffic and then it's gonna behave as if it was just a normal switch like how 6500 would behave okay this question I think if you have an issue with either to keep alive or the peer link anything connected just to switch to will not be able to communicate if the device is in the VLAN which is part of the V PC that is true so we'll take a look at the case exactly the the specific failure case is that that is going to happen in then can you please describe what are the functions the primary / the secondary we're going to talk about that okay so before we get into the specifics of the failure scenarios there's another additional design that I want to show here which is going to be our back-to-back V pcs so in this case we're going to be doing a VP scene from the 7 KS that are going down to the five K's so we're going downstream with a V PC here but we're also going upstream with a V PC so from the five K's up to the 7 KS now what is different about this design versus the other ones that we saw before is that normally on the 7 KS we would have 2 V pcs we would have V PC 51 and we would have V PC 52 where 52 is going down to 5 K 2 here and here V PC fifty-one would be going down to 5 K 1 here and here then on the 5 case if we were normally doing V PC upstream we would likewise have to separate V pcs will say that this is 71 which is here and here and then we would have 72 which is here and here this type of topology is not supported because again remember that the the logical result that we're trying to achieve from the physical topology is that we have a physical triangle so these are the V PC peers that's a physical triangle that we're trying to turn into a logical point-to-point channel but in the case of the back-to-back V PC if we have two V pcs upstream and two BBC's downstream it looks like two separate logical connections and basically just this is not a supporter topologies it's not going to work for forwarding in the data plane so what we actually have to do if we're doing back-to-back so we're doing V PC up and down we would say that there are two V PC's total so we'll say there's a v pc v that is on these two ports these two ports these two and these two then on the downstream switches we'll say there's VP c7 which is here here here and here so based on these number of lengths we can see that each of the devices has four links so it's it's going to be basically multiplexed on all these different possible ways from the ether channel load balancing point of view but from a logical view like a spanning tree we're basically just going to see one individual port channel so it simplifies the forwarding from a logical point of view but it makes the forwarding more complicated from a physical type of view and what we're trying to visualize really what's going on in the other configuration for this okay so let's go ahead and set this up and the first thing that I want to do is we're gonna start at the seven case and then we're gonna do the V PC down towards the the five case so let's go ahead and get on the command line here and what I've done is basically removed all the changes that we made yesterday so we can start from scratch and make sure we're not overriding any other configuration changes that we had made before so if we look at the show run I'll say show Ron pipe no more which means I don't have to wait for that that more output it's basically just default config here so I have the the VD sees that are configured but really the key point is that for the the interfaces there's no interface I'll configure same should be true on this guy if we say show run pipe no four no more if there's nothing configured on the the ethernet links here okay first step then I need to do is to make sure I've get basic layer three reach ability between the devices and yesterday's example we did this by configuring the m1 ports in a layer three port channel I'm going to show a different minor variation of this where the link is not going to be natively a layer three routed interface we're going to make it a layer two access port port channel then we're gonna use a layer three SVI in order to do the the keepalive between the neighbors so first on the the first seven Kay we need to turn the interface VLAN feature on so feature interface VLAN and we'll say that this is gonna be interface VLAN we'll say 40 94 so the last via then actually we can't use that one let's try 40 93 let's see which ones can we use show via and brief okay you'll see these these VLAN numbers these are going to be different the different reserved ones are gonna be based on the different platforms so 5k works a little bit differently than the the 7k does here let's try four thousand that's internal let's say three thousand hey there we go okay so it's it's basically an arbitrary number any any number as long as it matches between the two that's really only all that we care about okay so the interface VLAN 3000 the IP address here will say 169 254 0 71 slash 24 I could likewise put this in a vrf if I said vrf context we'll say this is to keep alive I could go to the VLAN say that this is a vrf member of keep alive and this is then gonna make sure that this layer 3 link is completely isolated from the rest of the data playing so no one would be able to send any traffic to VLAN 3000 it's used now it's exclusively just for the peer keepalive okay so the same thing on the other side let's a feature interface VLAN I have BRF context keep alive on interface VLAN 3000 I have vrf member keep alive IP address is 169 to 54 0 72 slash 24 okay then I'm going to have my actual links that are connecting to devices so on this side these are interfaces II 1 9 to 10 and these are now going to be switch ports hey remember that the m1 ports or the m2 ports on the catalysts or excuse me the Nexus Nexus 7000 those are going to be in routed mode by default so I need to put them into layer 2 switch I'm out yell then say that these are part of the channel channel group 3000 if I wanted to use la sepia we need to turn that on first so if I were to say mode active says I need to say feature LACP first okay so let's go back to the links channel Group 3000 mode active interface port channel 3000 is switch port mode access switch port access VLAN 3000 okay and then also I need to enable the links no shutdown so same thing on the other side let's say feature LACP interface II 1 1 2 2 these are switch ports they are in the channel group 3000 mode active interface port channel 3000 is a switch port switch port access VLAN is 3000 and then the physical links are not shut down and its turn our logging on here terminal monitor so if we now show IP interface brief for vrf here keepalive right now it says the link is admin down so I also needed to say you know shut a same thing on the first side interface VLAN 3000 no shut ok if we know and actually create VLAN 3000 show spanning-tree VLAN 3000 okay we should see that the port channel 3000 now is running this instance which it is okay this is the route port means that the other 7k is the root of the spanning tree for VLAN 3000 but really the goal is I want to make sure I can ping 169 254 0 71 from the vrf keep alive which I can okay so now I basically or 3 keep layer 3 connectivity we can create the VPC domain and then specify that that's going to be the keepalive destination so again in this particular example we are using back-to-back physical links for the keep a live link but it technically doesn't have to be it can be any link that is allowing us to establish layer 3 reach ability so it could be our normal in band data links like maybe going upstream towards the core I could just have an IP address and one of them and then specify that those are the other keepalive destinations doesn't have to be on the same subnet bow to a lava it's just layer 3 reach ability ok so next let's create the VPC domain will say VPC domain 7 for the 7 case first actually feature BBC so domain 7 my VPC pure keepalive destination is 169 254 0 71 I need to use the vrf keep alive and I need to change the source to my local address show Ron V PC ok same thing on the other side then we'll say feature V PC V PC domain match the numbers between the two of them the peer keepalive destination is now 169 254 0 71 the PRF is keep alive and the source is my address 169 254 0 71 so this should be the destination should be 72 there okay once this is configured again next thing we're gonna look at is to show VPC okay is the peer alive this tells me that I have basic layer three reach ability over the keepalive a segment for them now over the other keep a live link okay now I can move to the next step which is establishing the peer link so the pure link in this case is going to be on the f1 ports okay could be an m1 module but in this case it's f1 on these links e to one ought to two will say that the channel group is three thousand and one mode active on interface port channel three thousand and one this is the V PC peer link V PC peer links should be switch port mode trunk and spanning tree port type network and now shut down okay same thing on the other side interface e - nine - ten channel group three thousand and one mode active interface port channel three thousand one is the V PC peer link the switch port mode is trunk in the spanning tree port type is network and no shut okay if we now look at the show V PC what we should see is that there is going to be the election for the cisco fabric services and we should see that there's an adjacency it's talking about the CFS adjacency beer adjacency formed okay if we were to look at the show CFS or show CFS application we should see that V PC is using CFS over the physical Ethernet link again this is what we need to to exchange our control plane so we would not we would need to make sure that this is not filtered out under normal circumstances you cannot deny the CFS and the data plane but we just need to make sure that there is connectivity there between them okay so now we're ready to actually configure the the member ports so the pure link is up specifically in this case the member ports are basically going to be everything so these two these two these two and these two will say that this is V PC v going down to the five K's so specifically these links are going to be interface e2 11 to 14 and remember we want to make sure that these are shut down we're gonna shut all the member links down configure the V PC activate them on the primary let the control pain settle down for a little bit activate them on the secondary and then we should see the consistency check is correct over the pure link and then the V PC is actually going to start to ought to forward traffic so we shut down the member links will say that this is Channel Group 5 mode active on interface port channel 5 this is in V PC v ok the spanning tree port type is network and the switch port mode is trunk same thing on the other side here now so interface e to 3 to 6 should be shutdown channel Group 5 mode active interface port channel 5 is vb c v squid port motive trunk and the non spanning tree port type is network ok so now the 7 KS are prepped ok we need to now go downstream and do the same thing on the the 5 case so let's look at our our full config for this step-by-step let's look at this in just our text editor so we know that we need feature LACP we need feature VPC okay we then have our V PC domain I'll say this is domain 5 the peer keepalive destination is going to be the management link so we'll say 192 168 0 52 ok this is going to be on 5k 1 then we have interface II 1 1 2 3 V 1 1 2 3 here this is going to be for the pure link so these are switch ports they are in the channel group will say channel group 50 mode active interface port channel 50 is switch port mode trunk spanning tree port type network and the V PC peer link and now shutdown on the physical ports okay so this is prepping the the basic control plane of it again the pure keep a live link and then the peer link so let's say terminal monitor will turn logging on and then let's paste this in the first 5k on the other side it's going to be the same just the exception is going to be the peer keepalive destination is different so that's for 5k too we look at the show VPC we should see that the pier is alive and then once this converges the control plane pure adjacency is okay a 5k - is this secondary this should mean that 5k one when we show VPC is the primary okay Pierre and Jason see is up okay so now again we're ready for the actual data plane so data plane is going to be on e 1 8 to 10 on both sides so we'll say that this is going to be VP c 7 and for clarity that config it's it's probably a good idea to do some additional documentation and here like if we went to these links II 1 8 to 11 we could say like description this is for VP c 7 just so you know vp c 7 to the 7 KS or something like that now of course you don't necessarily have to do that it's just nice kind of for the the management afterwards to make it a little bit clearer exactly which vp c is used for what function okay so we have the Member links there this is going to be you know I will say no shut down that's fine because on the other side they're already disabled no shut down this is going to be channel group 7 mode active interface port channel 7 is a switch port mode trunk spanning tree port type network and VP c7 same thing on 5k - so interface II 1 8 to 11 no shutdown channel Group 7 mode active on interface port channel 7 this is which port mode trunk spanning tree port type network and VB c7 ok so now we're at our very last step where the links are disabled upstream on the 7 KS we're going to go to 7 k 1 which is the primary and activate these we should see that on the five K's downstream a portion of the V PC comes up not fully then once that is settled down we'll go to the second enabled those links and then we should see it's going to be the back-to-back VPC bidirectionally between them so on 7k 1 let's say on those links which are free show V PC and show port channel summary these are interfaces e to 3 to 6 no shutdown notice here we have V PC log message BBC 5 is now up ok these physical links are now members of the the channel bridge assurance is blocking VLAN 3000 on the poor channel which is probably what we want because VLAN 3000 is only used for our pure keepalive between the 7 KS so again when we're running spanning tree port type network we're using that bi-directional BP to you as a keepalive but also the result of that is similar to VT b pruning where it's automatically editing essentially dynamically the allowed list of the trunk saying that they're not sending me VLAN 3000 so I'm not going to send VLAN 3000 to them as well ok we will still send the P P to use in the control plane but no traffic is actually sent in the data plane so if we show spanning tree VLAN 3000 we should see on the port channel that this is now blocked because there's a bridge assurance inconsistency which is actually what we want to see here if we now show V PC we should see V PC 5 is up if we go downstream and show V PC be PC 7 is up on the 7k or 5k to you as well V PC 7 is up okay last step we have the physical links on the other side if we show show port channel summary these are links e 2 11 to 14 no shutdown if we show VPC we should see now our local ports are up if we look downstream DP c7 should still be up but if we look at the show port channel summary we should see that all four of these links are now part of there now members of the V PC or of the port Channel and if we were to look at the show spanning tree let's say for a VLAN one for example the 5k one is saying I'm the root of the spanning tree okay to clarify this a little bit more let's go up to the seven case and let's actually change the priority let's say spanning tree V and then one priority is 4096 then let's look at what the result of the output of this for show spanning tree is on all four of these piers if we show spanning tree V than 1 7 K 1 says I'm the root 7 K 2 should say that the pier link is the report if we look at the five K's they should see from show spanning tree B then one they see the single V PC is the report ok the pure link still needs to be forwarding this needs to be designated because if there's any control plane messages or there's any orphan poor data plane or multicast data plane that's actually used in V than one we will need to forward that over the peer link but the key here is that now from spanning trees point of view and from the rest of the data planes point of view this entire physical topology is logically collapse down into this with an eight-way port channel so 7 K at the top 5 K at the bottom so as you can imagine then from from the rest of the data plane operation this simplifies the network greatly because we have tons of different possible directions that we can be forwarding if any of the individual switches goes down we don't really need to worry about the failure because we should still automatically get sub second convergence if any of the individual links goes down likewise we basically have seven left over so we have essentially 80 gigs of possible throughput between the devices here okay so let's recap our full config here if we look at the seven KS and let's say show Ron pipe no more so again first we have L AC P and V PC on globally the features we then have the V PC domain and the peer keepalive so the domain the domain may are the domain number here it needs to match on the other V PC peer which is the other seven K in this case okay the actual address is here these are the switch virtual interfaces that layer three VLAN interface but that's kind of arbitrary it could be the management if we had separate physical boxes it could be a point-to-point layer three length could be a point-to-point layer three channel could be over any other route and infrastructure as long as I can ping between these two addresses that's all that I really care about okay so that's the particular SPI then we have the peer link if appear link again has to be a layer two trunk has to be carrying the VLANs that are in the V PC members and we would want it to be spanning sheep or type network so if we accidentally leave a VLAN on the link that's not being used then British urns is going to block that and make sure that we don't have additional unnecessary data plane traffic on the peer link because again peer like we want to keep that mainly just for the the control plane so then we have the final V PC member port since this is in our connectivity between the switches we want to say that this is layer 2 layer 2 trunk that's a port type network and that this is V PC v again this number has to match between the BBC peers the port channel number doesn't necessarily have to but again you really wouldn't want to miss match it it's it's just the config is clearer here when you match the V PC number to the port channel on numbers okay there's a couple questions here don't we need the Pierce which command for the single switches ID in spanning tree okay good question if we look at the 5k s and we look at the show spanning tree notice that we see the root bridge is the same value here okay between the different devices really the only thing that's different is what's the root port number the upstream device 7 k1 if we show run include spanning-tree this physically is the root bridge okay so out of those four devices this is the actual root the spanning tree protocol root but on the downstream five K's they see these links that are part of the V PC going to the other upstream pier as their normal route ports for spanning tree so it's actually a modification of the behavior of spanning tree here when we're doing VP sees that the V PC secondary is not going to be generating its own VP to use on the V PC member ports so again the key is that the regardless of where the root bridge placement is the root bridge could actually be the secondary V PC pier but it's the primary device that is in charge of basically maintaining the control plane so from the downstream switches point of view they only see one root bridge upstream where in reality that's not the case there's there's two switches one of them is the root one of them is not the root but there is an additional feature here we could say onto the BBC domain we say V PC domain 7 which is the pier switch on both of the switches if we say V PC domain 7 pier switch then I would want to change my spanning tree priority to be equal between the two of them so if we say spanning tree V then one priority is 40 94 or 4096 and we now show spanning tree V then 1 7 k 2 says I'm the root and you show spanning tree V than one and the other 7 case says it's the root so this is basically kind of a hack on the the spanning tree process that there's two root bridges but they're working in conjunction with each other because of this peer link to synchronize the control plane so it's not the normal behavior of spanning tree from the downstream switches perspective we're now receiving VP to use in all of the ports but they're all equal is the same priority and it's the same bridge ID that's that's coming in okay the reason why you would want to do this we're going to talk about this a little bit later has to do with certain failure scenarios where the primary VP CP R goes down the secondary becomes operational primary the primary comes back is reelected the root bridge then we have to go through the STP synchronization process in order to reconverge and in high availability environments you would not want that because you're going to have a temporary disabling of the data plane when you go through synchronization what you have to do in rapid spanning tree is to basically temporarily block all your downstream ports do the handshake on the root port agree that yes you are the route I'm downstream from you then you can unblock your designated ports and continue the handshake downstream which is the synchronization so it generally is a sub-second process but there is some measurable time that the convergence takes and especially in high availability environments of very low latency environments like if you're doing high-frequency trading you would not want to to add that additional latency to the the network control playing convergence okay but we will come back to revisit that again as well okay there's a question is back-to-back VPC a double sided VPC that's the same term right yeah double sided VPC is the same thing as back-to-back VPC there's another question do you think it's a good idea to use the SVI is I think you have to filter this VLAN from the VPC peer link okay yes you would want to do that and I actually did not do that in this example the SVI that I used on at the pier keepalive I don't want that to run on the peer link so what I would also need to do is to go to this port channel on both sides of the switches and say that the switch port trunk allowed list is going to remove VLAN 3000 because I want to use two separate physical infrastructures for the pure keepalive versus the pure link a pure link is for synchronizing the control plane here keepalive is just to make sure that we don't run into that active active split brain scenario there's another question as per cisco why do they say for double-sided VPC the number has to be the same for both sides and I have configured five in seven you could actually use the same VPC domain number so the way that I configured it here I said that there's regardless there's going to be two separate domains okay I said that there was vbc domain seven here and I said there was VPC domain five down here now what does this number really mean where is this information exchanged between the two switches okay they need to have the same number if they are peers with each other but what link are they using to exchange this information either using the peer link which is for the synchronization of the control plane the peer link there's two separate physical ones in this case there's a peer link between the seven case and there's a peer link between the five K's what's running over this is the cisco fabric services over Ethernet protocol so even though I do have layer two connectivity between the two separate domains even if the numbers were the same it's not going to make a difference because I'm not running c FS OE for the V PC process on those normal V PC ports is only on the peer link so I would say personally I would keep them as separate numbers just for clarity that each pair of switches you would just choose a new random number okay it doesn't have to really mean anything just so when you're looking at the config if I'm 10 on this side and I should be looking for 10 on the other side but I just use number 1 over and over and over everywhere in the network gets a little bit more confusing when I'm trying to do any type of verification or troubleshooting there's a question why would you use double sided V PC instead of 2 v pcs facing downstream what is the benefit the benefit is that it looks like one logical link from the perspective of the load balancing and from the perspective of spanning so when I'm sending frames north from the five K's up to the 7 KS I basically have 8 possible different decisions so there's four physical local ports on each of the switches then based on the ether channel load balancing I can choose either of those ok same for the southbound from the 7 case downstream towards the 5 case we just have more links to not to do the load balancing between it's basically the same as if I had to physical Chasse ease one upstream one downstream I connected them with 8 10 gig links and put them all in a channel that's the logical result of what we're doing here ok so it looks like just two switches from the the the logical result there's another question the pure keepalive could this also be set up as a layer 3 ether channel instead of an SPI yes it could ok we looked at that example yesterday with the other 7 case using the layer 3 port channel now there's a question why do you have to change the source for the pure keepalive links when you're not using the management link you have to change it if you just say pure keepalive destination and let's look at that config difference actually between the 5 case in the seven case on the 5 case if we show run V PC I said the pure keepalive destination is just this address ok this assumes that you're using management 0 and you're using vrf management if you're not using that that's when you have to specifically tell it if we show run V PC I'm saying this is the destination but the destinations not out management 0 I need to send it from this source in this specific routing table so vrf keep alive if we show IP route v RF keep alive and compare this to show IP route v RF management or just show IP route talking about the default global routing table we can see these are three different logical distinctions so only in the keepalive vrf do I have these 169 254 addresses and vrf management these are my management 0 addresses 192 168 0 and then I don't have anything configured yet in the global routing table here ok there's a question can we use M ports instead of F ports yes you can okay we'll see 4 4 V PC there's not really restriction for that the restriction is going to come in when we get into fabric path and then when we configure V PC Plus so VPC plus is the combination of V PC and fabric path at the same time and the requirement is that fabric path can only run on f1 or f2 ports on the 7k so it means that your peer link would have to be either f1 or f2 so in our case they're going to be the the f1 ports there's a comment here you can change the system Reserve VLANs in global config mode we could change those if we if we want it as well they're basically they're just arbitrary numbers as long as you have Mike documentation that you know what meaning is used for for for which it's fine again if we look at the show interface trunk like we saw yesterday we will see and it's actually different between the five case in the seven case show interface trunk you could see what's allowed here there's ones that are removed that are the the reserved VLANs so you could change these if you want to but in my case I just used a different number that wasn't reserved ok there's a question I guess this is similar to to VSS pairs connecting to each other now and z but instead of two control planes one prepare there are four control planes that's that's correct okay so with VSS basically VSS is choose 6500 chassés that are connected with the v SL v SL is like the pure link in the case of EPC okay the difference is that over the v SL there's basically an election process that they say one of them is going to be the VSS primary the other one is the is the VSS secondary like the standby so only one of the supervisors is in charge of controlling both of the physical chassis x' so even though there's two physical boxes there's only one control plane so when you met when you do management changes like you tell that in or you connect the console you only connect to the active supervisor okay in the case of V PC we have two separate control planes which again the means that if I tell that to one box I tell it to another they're just different interfaces I can make different changes so even though in this final result of the logical topology again looks like this where we have one upstream switch with eight physical links going to the downstream switch okay so this is the logical but we know that the physical topology is actually these four boxes with the full mesh of interconnectivity here so there's 1 2 3 4 control planes [Music]

Channel: IT-TALK IT-TALK

Views: 8,461

Keywords:

Id: j-kfVhnKVFc

Length: 66min 31sec (3991 seconds)

Published: Thu Nov 29 2018