Future-Proof Your Network: Upgrade to 10G with UniFi Fiber Gear

oh that was a big one in today's video we're going to talk about all things 10 gig how I upgraded my home network from these unmanaged gigabit switches here and I've got all sorts of stuff in this box you know the the standard issue number one eight Port Netgear unmanaged gigabit switch uh tp-link one here there's a Netgear Poe one at the front there this this gear is great it's cheap it's reliable but it's basic there is no VLAN functionality there's no management stuff and obviously it's gigabit speed Limited and so what I thought I'd do in today's video is take you through how I upgraded my home network to 10 gigabit using fiber I've done that using ubiquity unify gear and we'll go through all of that stuff throughout the video what's an SFP versus an SFP Plus what's a transceiver what cables do I need what types of connectors what sort of distance can I run all those questions and more we'll cover in today's video but for now out with the old and in with the new let's head down to the basement to get started [Music] you join me in my glamorous basement next to my what I swore at the time was a temporary server rack I built this thing five years ago and we all know there's nothing more permanent than a temporary solution but above it is my brand new comms rack and in there I've got a couple of unify switches and some Fiber stuff that we're going to go through in today's video but first I wanted to walk you through my network architecture now this may be different for your building but for mine this is what made the most sense to me there are some different schools of thought about how you should design a network and in fact if you ask two different people you'll get two completely different answers but I think most people will probably agree that you shouldn't daisy chain switches however that's exactly what I did I decided when I moved into this building I didn't really want to run Ethernet from all of the devices upstairs my primary workstation my backup server Wi-Fi access points some security cameras there's probably honestly about 15 or 20 different ethernet Jacks upstairs in this building I didn't want to run all that ethernet all the way down to this basement that would have been a hundred foot run times 15 or 20. so for one thing it was going to be a lot of expensive cable but another there's a lot of cable you know we're talking about something that's going to be this sort of size and I just I figured that that was going to be too much effort and so what I ended up doing was running all the ethernet for downstairs to this comms rack behind me tacked to the ceiling joist above my head and then put up through the floor to my home office and a couple of security cameras on the outside of the building here a couple of Wi-Fi access points downstairs too and then all of the ethernet for upstairs goes to a separate rack which we'll see later on in the video that pretty much mirrors this one almost exactly in terms of in terms of hardware and so for me that made a lot of sense because it meant I didn't have to run ethernet cable 100 feet times 15. but it did present some challenges in how to connect the two halves of the network together and so when I first moved into this building I decided that I wanted to Future proof myself so that I could go 10 gig one day in the future well that day arrived about a couple of months ago when I was having some network issues my open sense box which is in the rack just behind me over here decided that it was going to just completely drop off the network after many trials and tribulations I decided that actually maybe I should replace this with real networking gear not the big box of cheap unmanaged gear but proper UniFi switches with a management interface so I could actually figure out what was going on and get some visibility into my network so like I was saying figuring out how to connect the downstairs in my basement where we are now to the attic you know 100 feet away or so as the cable flies uh I I just ran a cat 6A cable and for gigabit speeds it was perfectly sufficient however when I started trying to use 10 gig over that copper cable I ran into some issues what I ended up doing in the end was running a pair of om4 fiber cables between this rack and the Attic So I have two 10 gig links between this rack and the one upstairs that gives me 20 gigabits of aggregated bandwidth which is it's complete Overkill at least for now I mean I thought I wouldn't outgrow gigabit slowly but surely though as I've started playing around with kubernetes at home doing a lot more stuff with proxmox and editing these YouTube videos I've started throwing larger and larger files around my network over time and I tell you I thought 10 gig was going to be completely almost a bit of a waste like I was almost throwing money away doing this upgrade but it's been really nice I'd almost equate it like upgrading to an SSD for the first time you just have so much overhead and so much bandwidth available to do multiple things at once whereas with a spinning hard drive it's a bit like Wi-Fi it's it's almost single threaded you've got you've got a limitation of just how much data can go through the pipe at any one time gigabits probably like a SATA SSD whereas 10 gig is more like an nvme it's it it's like come on come on bro show me what you've got is throw anything you can at it with multiple clients and with the proper routing with a managed switch you can do some really neat stuff so what equipment did I end up buying in the end then well I went with my old favorite unify a few years ago now I bought one unify Wi-Fi access point and here I am down the road I've got now four UniFi switches three maybe four unify access points it's safe to say I'm bought into the ecosystem I really like the way they separate the configuration of the physical device with a remote control software now those of you in the Enterprise world that's completely commonplace but a few years ago when I bought that first unify Wi-Fi access point it was a completely new concept to me and you know we're all used to the 192.168 1.1 you know IP where we log into the web UI of the specific device and configure it that way well with unify gear you you tether it to a remote control software and then that pushes out all the updates over the over the network to those remote devices that controller can be self-hosted and in fact it runs for me on one of these boxes just here for many for many years actually I self-hosted it on a linode VPS but now I've got so much stuff locally I actually brought the controller in-house it's just running in a container from Linux server.io now what about the actual Hardware that I bought in the end well I went with a usw aggregation switch that's an eight Port 10 gigabit SFP plus fiber switch now SFP plus means that those ports are 10 gig capable provided you have a 10 gig transceiver and we'll come on to transceivers later on below that is a unify 24 Port Poe switch and that's for me to run the security cameras on my Wi-Fi and that kind of thing now that has 16 Poe plus ports eight non-powered ports and two one gig SFP ports so the difference there SFP plus is 10 gig capable and SFP is one gig capable now I'm powering a couple of Wi-Fi access points and about six or seven different security cameras off this switch so my power budget and a really nice thing of a managed switch like this is it you can actually see how much wattage each device is using so each each access point in each camera draws roughly 5 watts in my experience some some will be more some will be less so with that what what did I say like seven maybe nine devices I'm nowhere near the total 95 watt power budget that that Poe 24 switch offers now if you need more Poe power than that they make a pro version of that switch which has 10 gig SFP uploads a pair of 10 gig SFP uplinks as well as a much higher 400-ish watt Poe power budget so why did I end up with this specific kit then well let's take a moment just to discuss the other options in the 10 gig space from unify they offer a few switches with 10 gig most notably is the UniFi Flex XG now that's probably the only sane option for home users looking to go 10 gig that uses copper so it has RJ45 plugs rather than needing fiber cables and fiber transceivers to get those 10 gig speeds it can also be powered over Poe so the actual switch itself supports Poe in over its gigabit Uplink port in addition to its four 10 gig RJ45 ports even if I do offer one other device that does 10 gig over copper and that's the usw Enterprise that thing has 24 10 gig ports a couple of 25 gig sfp28 ports but it costs thirteen hundred dollars now let's talk a little bit about the mechanics of how to actually connect the two racks together originally I ran a cat6a cable so that I could do the 10 gig over ethernet that way and I use the cheap cable tester to check that the cable was good it showed the light sequentially one through eight and everything tested just fine and I used that cable over gigabit speeds for the last three or four years but as soon as I started trying to run 10 gig over that cable that's when I started to have issues I was having random dropouts and not only of devices upstairs which was the confusing part for me you'd think if the link between downstairs and upstairs was flaky only devices upstairs would be affected but no even though my DNS server and all my networking gear is down here in this rack it was making things everywhere in the entire building even stuff in this rack misbehave it was a very strange time and so I ended up buying a quote-unquote real cable Tester the Klein land Scout Junior and when I used this cable tester it showed me that one of the Twisted pairs had a bad termination or a break somewhere in the cable and sure enough I went and re-terminated the cat6a cable and suddenly 10 gig was stable magically for a few days it was stable enough that I was able to go and record an episode of my podcast self-hosted and extol the virtues and say yay look at me look how great the 10 gig upgrade is over cat6a yes anybody can do it over copper it's wonderful all the naysay is saying copper runs too hot and is unreliable and a bit flaky they're all wrong they don't know what they're talking about well the very next morning after I had recorded that episode it's almost like computers know sometimes the flakiness was back and that was it for me I was like right I cannot be doing with this this has been going on now for at this point for two two or three weeks of just consistent unreliability in the network and I just decided screw it this is it I'm done with copper I'm gonna go fiber it's what I should have done in the long run it's the most future proof and by the time you work out the costs it's actually only about 10 or 15 percent more expensive it's probably worth just recapping very quickly the terminology at play here so SFP is gigabit speed SFP plus does 10 gig and if you really baller SFP 28 does 25 gig speeds we'll get to more of the terminology about connectors later on but before then it I just wanted to talk about why I didn't end up going with the poe 24 Pro switch that unify offer that thing has a much bigger power budget 400 watts so if you're powering multiple Poe switches perhaps or other more power hungry Peri devices that's going to be the one for you the other thing to think about is the pro switch comes with a pair of 10 gig SFP plus Uplink ports compared to the non-pros SFP ports and this is where traffic flow starts to come into the equation you've got to think about things like where is my NVR going to live because those cameras are going to generate a lot of traffic what about my primary server that's pulling down things from the internet near enough line Speed Most of the time where's that going to live that it's going to reduce the number of hops for its primary traffic use and so it's all about minimizing hops that's the that's the name of the game minimizing the Hops minimizing the amount of traffic that traverses the different interlinks between the different switches in the building and spreading out the traffic as evenly as you possibly can that's where the planning phase of building out a home network really starts to come into its own if you skip that phase you'll end up regretting it later on because you didn't think about which devices were going to be pulling down the most bandwidth in a particular spot so I've taken up one of my precious SFP plus ports connecting the internet directly into the aggregation switch now at first glance that might seem a bit silly but when you think about how the traffic flows from all these different devices from all the devices upstairs into the aggregation and then out to the internet most of my 10 gig devices live upstairs my workstation my laptop my backup server back kind of thing and so in fact using up one of those SFP plus ports as a RJ45 capable gigabit port for the internet made complete sense so whilst I might only have one actual 10 gig device connected to my unify aggregation switch down here I do have a couple of other ports taken up with the fiberlinks going up to the attic now I used om4 fiber cables for this purpose they were 50 meters long and cost about 60 dollars each om4 is a type of multi-mode fiber now multi-mode is typically used over shorter distances and by short I mean less than 300 meters and in most residential settings that's going to be more than enough it also supports speed of 100 gigabit up to 150 meters your ISP might use what's called single mode fiber to send the internet from their Exchange building to your house and that's designed to go multiple kilometers let's just talk briefly about client devices next to me I have my primary server and in it I have a connect X3 card from melanox I got this used on eBay for about thirty dollars and then spent a further 20 on the 10 gig SFP plus transceiver required to make it function with the switch I then connected the client device to the switch using a much shorter om4 fiber cable and I've been really pleased with the outcome and this is where one of the surprising things about doing 10 gig over copper really caught up with me I thought the client devices would be the cheap part with copper didn't turn out to be the case for me I tried a couple of used 10 gig cards off eBay but both of them were faulty one of them the fan literally didn't Spin and the other one just wasn't detected by my version of Linux I'm running proxbox with a modern kernel just wasn't seen I had no such issues with the connect X3 by the way that was just plug and play the 10 gig base T cards base t meaning ethernet in this context they cost me anywhere from 90 to 100 each which when you look at the fact that a connect X3 card with a transceiver is about fifty dollars you can see very quickly that fiber doesn't start to look quite so expensive anymore we talked a little bit about the overall feel of a fiber Network already I just can't put my finger on it just feels snappier somehow um but there are some downsides to be aware of the most notable of which is the fact that you can't terminate your own cables you have to buy pre-made pre-terminated cables that can get a little bit expensive compared to running and custom length ethernet cable crimping your own ethernet cables the other downside is that the terminology can seem a little complex you know there's a lot to think about you know SFP plus direct attach all sorts of other stuff LC to LC all sorts of other terms to worry about but if you take your time do a little bit of research it's not too bad now the other thing to think about is used Enterprise gear and in a residential setting it can be really off-putting to use some of that gear it has a high energy draw the fan noise can be unbearably loud I'm not going to sit here and say that unify gear doesn't have a tax associated with it especially when compared with something like a microtix switch and we all know the power of an ecosystem sucking you in so let's head upstairs and take a quick look at the gear that I've got up there well I made it upstairs safely next to me I have another rack that looks mightily similar to the one in my basement and that's completely by Design so what I wanted to do was mirror the hardware I'm a huge fan of not having to think too much when stuff breaks I tend to use the same CPUs and motherboards in my servers where I can and just standardizing equipment I find is a really great way to reduce the cognitive load of maintaining Hardware so what I've done is I've got the usw aggregation switch on the top as a pdu actually at the top the black bar then the aggregation switch then a gap then a patch panel and then the poe 24 underneath that that layout mirrors identically the rack in the basement the only difference for the rack in the basement is it's suspended off the ceiling joists and then I have a couple of servers underneath now this enclosure that's here this is something that I made myself there are some small rails in the side here you can buy these off Amazon there'll be a link in the description down below and these are made by Gator works and this is a 6u height rack and I had to work with the the sloping height of my ceiling to make things fit in here properly and so buying something off the shelf just wasn't really an option networking gear is quite shallow so I was able to get quite a decent amount of density in a small unit by building it myself out of some plywood now one of the biggest changes I made throughout this whole network revamp project was switching out my patch panel so in the old days I used to use this thing I used to have one of these in the basement or one of these up here and this is a punch down patch panel so on the front it has a 24 RJ45 ethernet Jacks rated at cat6a that was important to me obviously with the 10 gig aspirations I had when I when I moved into this house not a biggest fan of them having moved to keystones for my for my new equipment and the reason for that is let's say I wanted to move this cable here from port number one all the way across to port 13 for whatever reason I'd have to physically undo this metal clip that's here and then pull the cable out of the termination to move it and then re-terminate it over here doesn't sound like a big deal but it's kind of a pain the other problem is I can only use ethernet ports in here well with a keystone situation you're able to actually remove things much much more easily and so if we look at this thing here it has a little clip just on the top here and it slots directly into one of these square holes in the Keystone patch panel that's just here and you can get all sorts of different things to go in here so here I have an Ethernet keystone jack and here I have an LC fiber Jack so I take one of my fiber cables and I can just plug it directly into the keystone jack there's just a pair of lenses on either side it's just a pass-through essentially and I find it works really well they make all sorts of other stuff for Keystone Jacks you might see them in wall plates for example um they make HDMI you know ethernet fiber audio connectors all sorts of stuff so in in a home environment where you've typically got lots of stuff going on you don't necessarily need all 24 Jacks as ethernet you might want a mixed use a bit like I have some Fiber some ethernet some HDMI Etc I find the keystones just so much more flexible and installing them is really easy so if I take this this Fiber One in here for example can just reach round the back and you know it's a little tricky one-handed but I'll make it work and then I just push I get it I hook it in at the bottom I pull it in and it'll click ready that's done installation complete now the situation with ethernet tiny bit more complicated you have to punch down your eight Twisted pairs your eight strands of four Twisted pairs I should say into the back of this keystone jack then put a couple of little plastic protectors over that cable and then put it into the Keystone hole instead but let's just go back to the original scenario that I talked about with the with this guy let's say I wanted to move from Port 1 to Port 13. well be really easy because all I'd have to do is get a tool in now like a flathead screwdriver push down on this little tab that's holding the Keystone jacket in place and then move the entire assembly from Port 1 to port 13 rather than re-terminating anything so that's a huge benefit of using the keystones now what about when we want to actually start terminating and connecting fiber into our switches so we've got the fiber up from the basement now successfully that's what these blue cables are by the way well a couple of them anyway a pair of these run directly to the aggregation switch down in the basement where the internet comes in remember um and they run directly into these transceivers now these transceivers are probably the most complex part of building out a fiber Network you're going to want to follow a couple of rules typically when you're doing this and sticking with the manufacturer of the switch so in my case unify they make these transceivers a two pack of these transceivers costs some are in the region of 38 dollars depending on where you buy it from so shipped to my door I paid about forty dollars so each Transit was about 20 bucks all you need to do to install them is they have this kind of what looks almost like a pcie connector on the back like a little circuit board with some gold pins so what you need to do is take out the plastic plug that's in the switch then line up your transceiver with the slot you want to use and just push until you hear it click that's now installed it's as easy as that and so then the all that's left to do is to start connecting in the fiber now right here I have a brand new om4 this is a five meter cable the length isn't important as we discussed anything up to 300 meters in fact would be supported in this scenario fiber is obviously a light based transmission mechanism and because of that it's quite susceptible to dust now one of the things that you might not know and I certainly didn't when I was doing this for the first time is that you end up with lots of these little white caps that go over the lenses at each end of not only the cables but also the transceivers so you notice there was a black dust cap in the switch itself that was to protect the kind of the contacts the pcie kind of connector it's not pcie I say that so that you understand what it looks like there's the little black plug that came out of the switch there's also a little black plug that goes into the transceiver to protect the lenses with inside the transceiver and then there's also a pair of little white plastic plugs that go on the end of the cable to protect the end of the cable from getting Dusty too now it's time to actually plug in a fiber cable it's a really satisfying click let me tell you now uh the other thing left to do is obviously connect it into something on the other end so I'm just going to use the the keystone jack that we put in a few minutes ago much like the transceivers and the cables this has a couple of white plastic protectors I'll remove them from both the lens of the keystone jack and the end of this cable and then I'm going to plug it straight into the keystone jack and so that's how we go about actually connecting different fiber devices together if we were wanting to connect this directly to a server we'd obviously have another fiber cable connected into the back of the keystone jack and then running to whatever device that we were connecting to now I mentioned a little bit earlier about sticking with the brand of transceiver for the switch to have the best luck in the switches themselves as I said I I've stuck with the unify transceivers but I've had pretty good luck with these Intel transceivers in the client devices themselves in the melanox cards that I mentioned earlier the connect x3s and also my solo 10g Sonic Technologies SFP plus Thunderbolt 3 MacBook dock that I'm using to connect my MacBook into the 10 gig ecosystem that I've built here I'll put a link to the model numbers of these Intel transceivers I've been using down below but it is unfortunately just a case of trial and error so pay attention to your Retailer's return policy before you go and drop a chunk of change on one of these things so that just about covers the fiber side of things so far as transceivers go um I should also note that transceivers are the limiting factor when it comes to speed you've got the switches capability you know SFP SFP Plus sfp28 these transceivers are rated to those particular standards but the cables themselves are capable of all the way up to 100 gigabit so if I was to get a switch capable of 100 Gig and a pair of transceivers capable of 100 Gig I could reuse the fiber that's there and that's one of the things that's really nice compared to Copper if I was trying to use copper to go anywhere near 100 Gig it would just laugh at me and catch on fire whereas this fiber upgrade will last me dare I say forever is that a Bill Gates 256k ought to be enough for anybody moment who knows maybe I'll eat my words in a couple of decades but certainly for now it's plenty now what if we wanted to connect copper into the fiber ecosystem well that's where the RJ45 transceiver modules come in so again I purchased a ubiquity one here an RJ45 one gig transceiver and this is what I'm I'm using I bought a pair so I got a backup a spare if you like um this is what I'm using in my basement to bring the internet into the building so again it's got the little you know gold contacts on the back and then just an RJ45 plug on the front and installing it is exactly the same as installing a fiber transceiver just push it into the switch and wait for it to click boom like that and then all I do is connect an Ethernet cable in and I'd be good to go now obviously this is going to negotiate at one gig speed but if I bought a device that was 10 gig base T like maybe a Mac Mini for example this is exactly the device I would need to get to connect that into ethernet over you know a cat6a cable over a short distance you should be fine and they make these in one gig and 10 gig it's worth noting that these switches Auto negotiate at certain tears if you like so one of the nice things about the flex XG switch is actually that I forgot to mention is that they Auto negotiate at one two and a half five and ten gig speeds my Cal digit ts4 dock that I connect my MacBook Pro into has a two and a half gig network interface and in fact I think a lot of consumer networking gear is going to head towards that two and a half gig copper standard this switch the the aggregation only does one gig or 10 gig there's no middle ground so if you're looking for a two and a half gig switch the flex XG might be a better better shot for you I think the last thing to talk about are the direct attached copper cables so I'm about to break my network for you uh completely it's okay everybody else is in bed these are the direct attached copper cables and this is the simplest form of connecting SFP devices together so these are one gig direct attached copper cables and they look an awful lot like one of the transceivers don't they I mean the form factor is exactly the same they've got the pins on the back the only difference is I can't actually remove the cable from this so they're directly attached to the transceivers hence the name the upside of that is they're really cheap in comparison the downside is well you can't really run it very far I think they're limited to a few meters at most so they're great if you want to connect together a server that's you know a couple of meters away from a switch or something like that you don't need transceivers necessarily for that um but for anything over I think it's like five or ten meters something like that you're going to want to start looking at transceivers and proper fiber cables I think it's time I put the direct attached copper cables back into the switch before someone texts me that Plex is down so I think in general 10 gig for your average home labor and self-hoster might on the face of it seem like Overkill but I can tell you having had this set up now for the last month or two it's really really nice to have that bandwidth overhead of when someone's streaming a really high bitrate movie in Plex on or jellyfin on the TV and then I want to do a massive file transfer whilst I'm editing a YouTube video and I'm downloading from the internet at Full Line speed and doing a backup at the same time with ZFS across the network it's it's just really nice that all of those things feel fast the bottlenecks are no longer the gigabit connections the bottlenecks now typically is by spinning hard drives in my various systems it's a bit like Factoria I feel like the bottlenecks have all moved around a bit like I'm going to have to upgrade my iron Supply soon to you know replace my spinning rust with ssds in certain strategic places to make file transfers that little bit snappier I've been trying to think about whilst I was writing this video trying to think about reasons why people wouldn't want to do 10 gig at home with fiber and for me the only real reason is the fact it costs it does cost more than going gigabit but there's no reason why you would need to buy four switches like I did in a in a normal home so other than sheer laziness like me not wanting to run so many cables down to the basement I think for most people just buying something like this 8 Port 10 gig switch and having your workstation and your Nas and a couple of other devices may be connected directly into it it's going to give you a great upgrade overall just a great upgrade to your network and be a really nice starting point have an SFP Port going back to your main switch or or even use the SFP plus to an RJ45 cable you know those are all perfectly valid options most small businesses don't don't have a network that's quite as redundant and and robust as this so there's a lot to like there is a lot to like and I think um anybody looking to do 10 gig would be well served looking into the unified ecosystem if you're a microtic guy or you want to do something else with some old used Enterprise gear let me know down in the comments what you chose to do I'd love to hear from you and until next time I've been Alex from ktz systems thank you so much for watching

Channel: KTZ Systems

Views: 38,891

Keywords:

Id: nlB73DqNFxY

Length: 33min 9sec (1989 seconds)

Published: Wed Apr 12 2023