Budget 10Gbps Networking

if you're interested in learning how to add 10 gigabit networking to your network environment for less than 250 bucks then this is the video for you stay tuned welcome to crosstalk Solutions my name is Chris and in this video we're gonna be setting up some 10 gigabit networking for less than 250 bucks and we're going to be doing it with this mikrotik CRS 305 def 1g - 4 s + i n switch now that model number is quite a mouthful but the switch is actually a really nice little switch and it's very very cheap for what it does ok so before we get into that though let's take a look at the overall architecture of what we're going to be setting up today so basically I'm going to be doing 10 gigabit networking between two servers they are both abouttwo 16.04 servers however this should technically work with any server or computer that you can get the 10 gigabit network card installed into so wherever the card has drivers that will work this should work just the same so we have one server here and one server over here on the right and then they are connected with fiber into the sfp+ ports of the mikrotik switch that mikrotik switch is powered by a tow 2.3 AFP owee and that is a Gigabit Ethernet port that goes right into my unify switch I have everything set up on its own separate VLAN which is VLAN 88 and I have both my computer here that I'm recording this footage on as well as the mikrotik switch set up as untagged ports in my main switch that are all on VLAN 88 and I chose via an 88 just as an arbitrary number because the default network of the mikrotik switches is 1 81 68 88 / 0 / 24 so it doesn't have to be and we'll go over all those details when we actually get into the configuration and setup of this device but for now let's talk about something very very important maybe the most important thing that I'm gonna cover in this video and that is the correct pronunciation of mikrotik so mikrotik is a lot vien word and if you were actually Latvian you probably pronounce it me critique okay so that is the correct the official correct pronunciation of me critique but it actually depends on geographically where you're located in the world so for instance in the UK and in Canada they don't say me critique they say me throat tick and then here in the United States because of course we have trouble pronouncing words from other countries we say mikrotik alright so I'm gonna be saying mikrotik in this video just cuz it's easiest for me but I did do a quick Twitter poll and you can see here that after ninety four votes seventy seven percent of people agree with me that the correct pronunciation is mikrotik however the official pronunciation technically it should be me crow tick um I might say all three of those and maybe even some other variations throughout this video okay so let's take a look let's take a closer look at the CRS 305 one g4s plus i n so this is a five port desktop switch with one gigabit ethernet port and four SFP+ ten gigabit ports one other thing that I really like about this switch is that in the back it actually has redundant power so you can plug in two separate DC power supplies that range from 12 to 57 volts and it will take two different ones so you can plug these into two different circuits and then if the power goes out or one of those one or other of the circuits blows technically you should be able to keep this device online and it even has a nice little handy cable management plastic piece to handle two power cords coming off the side so it's a little tough to find this switch however I do see it available at Baltic networks will not available but available for pre-order for a hundred and twenty four dollars and 25 cents that's USD of course and it says it's available by the third week of May and I'm recording this on April 10th okay so this thing is backordered pretty severely I know when I got mine I probably ordered it like November or December of 2018 and I received I received it like sometime in January I think so probably about eight weeks if you're gonna order one today till you actually have it in hand so let's talk about the cards that I so the cards that I chose for my servers and this was actually on the recommendation of Tom Lawrence with Lawrence systems so thank you Tom for this recommendation I went with the Chelsea oh and 320 II this is a dual port sfp+ network card and I got mine off of ebay because I found him really cheap on eBay they were $29.99 each yeah here we go so this is the exact one that I got $29.99 looks like they have only one left if you wanted to get one and they took they actually came relatively quick they came in about a week now the problem with this card and I didn't realize it until actually I had received the card is that the bracket was a low profile bracket so it did not fit in a full-height PCI slot even though it does say on the it does say it's a full bracket card on the listing but the ones that I received were like a low profile bracket so I actually had to remove the bracket entirely and then just sort of sit it loose in the PCI slot to get it to work with the server so just keep your eye out on the form factor of the cards if you're gonna buy one now they do make sort of brand new these like an older generation Chelsea oh 10 gigabit networking card they do make newer cards however the newer ones are like a hundred and sixty six dollars on Amazon so way more expensive and totally blows the $250 budget for this project but if you are going to be installing this into like a production server in a production environment then I would probably go for the latest generation card just because it's gonna be supported a lot longer and again links to all of this stuff down in the description below but now let's take a look at how I connected the card now I chose fiber and the reason that I chose fiber was because I happen to have four SFP+ modules sitting around that I had bought a long time ago in fact you can see here it says I purchase item on November 8th 2017 and when I bought this item I think I bought like a couple 10 packs and I had a bunch left over I had like four of them left over which was perfect for this project because you need one sfp+ module in each of the servers and then two to connect the fiber to the switch itself I went with the 10 GTECH for ubiquity or I should say the ones that I used were the 10g tech for ubiquity SFP+ modules they actually did work perfectly fine in the mikrotik switch but they also make a 10g tech for you for mikrotik sfp+ module as well it's exactly the same price either one of them is gonna work and perfectly fine when you're using SFP+ modules you need to get fibre optic cable and what we want are the LC 2 LC multimode fiber cables here you can see them on Amazon in a variety of different lengths and a variety of different colors I have one 1 meter length that's the one that doesn't stretch all the way over to here and then I have one here that is a 3 meter length LC 2 LC multimode fiber cable however using fiber is more expensive than the alternative which is using a DAC cable or direct-attached copper cable now what a DAC cable is is essentially the SFP module plus the cable in one unit right so you see here we it has the ends on it these ends that are on this cable right here it's direct attached copper meaning that you don't need to purchase a separate SFP module this is built in to the cable right so basically you just buy this one cable with these DAC ends on it and it's much cheaper so whereas the SFP module is $20.99 and you need two of them so you're already up to 42 dollars plus the 14 dollars for the cable that's $56 total for that setup you can get one DAC cable for $18.99 that's actually a three meter deck cable you can go a little bit cheaper than that I think $15.99 if you get a 1 meter deck cable and the Dec cables come all the way up to the 7 meter length and the 7 meter DAC cable is about 42 bucks ok so let's take a look at the overall price in here now this is what I paid for everything I paid I ended up paying with shipping 141 28 dollars for the mikrotik switch I paid 60 898 for two of the Chelsea on3 20 e ten gigabit cards and then I paid 83 96 for the 410 GTECH SFP+ modules and I bought 28 dollars and 58 cents worth of fiber patch cables for a total of three hundred and twenty-two dollars and 80 cents shipped to me all done everything included of course that's not including the servers and stuff but we're going to assume that you already have those now the cheapest that you can actually get away with if you went with the DAC cables instead of the fiber setup is one hundred and forty one dollars and 28 cents for the switch 68 98 for the - Chelsey oh 10 gig cards and then 31 98 and that's going to be two of the one meter dat cables and the total price there for all of that equipment is two hundred forty two dollars and twenty four cents and if you go for the three meter deck cables you're gonna be almost exactly right at 250 bucks all right we're going to get to the configuration of the micro ticks which in just a moment however before we blow away the configuration completely let's test this out using iperf so here on the left hand side I have my one server which is 192 168 88.4 and we're gonna put that into iperf 3 - s server mode so that it's listening for connections from clients and on the right hand side here I have 180 168 88.1 hundred and we're gonna say iperf 3 - c 1 i2 168 88.4 to run tests against the iperf server that is in listening mode so let's go ahead and press enter boom and there we go so you can see here down this middle column that the bandwidth that we're getting for this speed test is 9 point 4 gigabits per second or about as close to 10 gigabit networking as you can hope to achieve so really really nice alright so let's go ahead and now blow the configuration out of the water we're gonna factory default our mikrotik switch and and then go from there so here I am in the micro tick or micro ticks which we're gonna say system reset configuration reset configuration okay now at this point I'm going to run a persistent ping the switch and we can see that it is actually already no longer responding to ping so it is down completely now by default mikrotik equipment comes configured as 192 168 88.1 with username admin and no password however it doesn't have any sort of DNS servers it doesn't have any sort of gateway set ok so that means that once you factory reset it or when you receive it you're not going to be able to connect to it from other networks because it doesn't know how to go anywhere else other than 192 168 88 0/24 so what you need to do is configure your own computer into that same network there we go so you configure your own computer into that same network so in my case I just went dhcp and my computer was configured as 192 168 88.1 Oh to the Gateway of this network is 180 168 88 254 and I saw in the background here that the router OS has popped up and we are now able to log in with the default credentials which should be admin and no password there we go ok now there's actually two different ways that you can connect to mikrotik equipment one of the ways is by just connecting directly to the IP address of the equipment in this case 180 160 and 88.1 at the default IP address however you could also use a program called wind box so I'm gonna run wind box this is free to download from mikrotik and if you come over here to win box you can just click connect right off the bat as long as you're in the same network because it already has 182 168 88.1 is the default choice with admin and no password you can also click on neighbors and hit refresh and it should find any mikrotik devices on your network so in our case we're just gonna click connect and ok and there we go so we are now in the mikrotik router switch if we click quick set we now have all of these options if you're in the GUI you can also just click quick set up here in the upper right hand corner and you get those same options however we're going to use wind box here so quick set we're going to leave mac-address alone we're gonna leave it in bridge mode we're gonna leave it static we'll leave it at 180 168 88.1 however if you want to put this device into your own network you want to use your own land settings here so the IP address the netmask and the Gateway as well as the DNS servers so again no gateway to begin with let's go ahead and add one 192 168 88.2 54 in my case and then we're gonna say 192 168 88.2 54 for the first DNS server and let's add a second backup DNS server of one not one not one dot one down for the router identity you're gonna want to give that router a name I'm gonna call it ten giggity and then for the password you want to create your admin password make sure it's something strong and something that you're not going to forget and hit apply followed by ok from here we can go to tools and go to ping we should be able to ping out to the internet 4.2.2 - let's go ahead and start and there we go so we're getting replies from 4.2.2 to the micro ticks which can now see the Internet as far as setup of the micro ticks which putting in a Nero network and just taking all of the rest of the defaults is good enough there's a ton of stuff that you can actually do with this switch but that's way beyond the scope of this particular video so as far as configuration goes we're completely done with the mikrotik switch if you click on interfaces here we can see we have Ethernet one and then sfp+ one two three and four of which 1 & 2 you can see that there's some traffic being passed if I open up sfp+ 1 and click on the traffic tab let's go ahead and expand this a little bit bigger so here's SFP+ 1 let's go ahead and run that same speed test again or the same iperf test and let's watch the graph move here run and boom there we go so immediately you can see that it jumped up to 9.8 gigabits per second on the transmit all right there we go test is done let's try it the other direction as well so I'm gonna say control see over here on the second computer I'm gonna say iperf 3 - s and then over here on the first computer iperf 3 - c 102 dot one sixty eight dot eighty eight dot 100 enter and yep once again you can see 9.41 gigabits per second okay so the only other thing that you're gonna have to do is get those cards installed into your server now with the boot - it was super super easy basically I just put the cards in the server and Ubuntu 1604 already had the drivers for it so it came up right away if you look at these Ubu two servers and we say Hat Etsy Network slash interfaces here you can see the one that I configured statically right so I just made this a static if you want to check for the card you can say LS PCI grep Chelsey Oh probably capital C here there we go so we can see Ethernet controller Chelsey Oh communications T 320 10 gigabit dual port adapter one final troubleshooting tips for finding network adapters in Abood - you can always do cat /proc / net / in dev and that's going to show you the list of adapters available in your server so in my case we have the loopback or the local area of the local address we have e n P 1 s 0 this is the first sfp+ card then we have e n o p0 s 3 1 F 6 that is our Ethernet adapter and then we have renamed 3 which I don't know why it does this but it actually did this in both of my servers the second port of the two port Chelsey o 10 gig network adapter was called rename 3 so I'm sure you could rename that somehow if you want in our case we didn't but then in the FC networking interfaces the only one I actually set up the only one of these that I actually set up was PN P 1 s 0 and you could just sort of fiddle around with those to see which ones are passing traffic to figure out which one is which in your own server ok so there you have it let me know what you guys think this project down in the comments below I'd be curious to hear your thoughts also how you pronounce a mikrotik yourself I'd love to hear about that as well okay if you enjoyed this video please give me a thumbs up if you'd like to see more videos like this please click subscribe my name is Chris with crosstalk solutions and thank you so much for watching you

Channel: Crosstalk Solutions

Views: 176,944

Keywords:

Id: 740wIinb4-Q

Length: 18min 11sec (1091 seconds)

Published: Thu Apr 11 2019