My Home Virtualization Server Running pfsense Inside of Citrix XEN Server & Autostarting VM's in XEN

so a few people asked about doing a home virtualization project you know something small scale and not too big like the Dell our 7:10 server we did and this is actually something I came up with well because it was on sale I won't lie and I have to reference the paper I hate make sure I get the numbers right I we went over to Micro Center and they had a FX 8320 Black Edition AMD processor which is a nice 8 core processor and actually does virtualization really well and grabbed an asus motherboard happens to be a m5 8780 lm+ they're pretty basic stuff here very consumer grade and it's really interesting because not only was it inexpensive it was on sale way she got this processor with the yeah I do a mail-in rebate you know it's Micro Center but the processor was 80 bucks and had a $40 million rebate and I think they had a deal where you bought it as a combo of micro Center and you got the board for like another $20 so really inexpensive very affordable you can find some of these deals online as well and then we actually had this case Steven bought it and never used it well he did he nicked it at the top so it is used technically but this is a third will take the land light land box light case and this is actually going in my house so my server at home is a little dated and is well by PF students at home is really old and my server that runs plex is barely able to keep up it when it won't do anything more than ten eighty and ten eighties pushing it to the limit so I decided to build a virtualization what was then again for a couple reasons we'll explain here and a software demo of how this all works but I wanted to virtualize all the things I do so the server's at home and it's more than one now and this is gonna bring it down to one server what it's doing is I have stuff that runs backups I have stuff that runs my plex and then I have a separate box for pfSense this is all-encompassing so we actually going to boot Zen we put a couple network interface in here so it's going to connect to my cable modem at home and it's that yields can have a virtualized version of pfSense inside of here so server will boot up and we're going to set it to auto boot to auto start so you know case power loss and nobody's home the kids will still have internet and the wife because that's important but I wanted something really compact and that should work with us so this is gonna be kind of a little project and I load it all up and then I'll do the video well it's already built now some do the video now on how all the software works is that's it that's the real nuts and bolts of all the virtualization is showing you how the software works and how we got pfsense virtualized and how you set that up there's some things you have to do and that's we're gonna cover to show you how to get all this working so well here's the box we got that part covered now well if you couldn't tell by the orange tiles that video was done about a month ago before we swapped the orange tiles down and that was on purpose well the tiles weren't purpose and the delay I shot the video because I wanted to put the machine in I wanted to use it for a while I wanted to load it all up and really test it and see if I had any hiccups problems or anything and it worked well but I'm gonna cover a couple little things related to it and making sure so if you decide to set this up in the same way I did at home where you virtualized pfSense Zen Orchestra Plex and your backup server and on the other miscellaneous servers you do in your home on a less lesser powered machine I'll go over how I did all that and set it up so I'm gonna start with what's right here the internet comes in there's only two network cards in this virtual machine the land box that I showed you and beginning here my internet provider currently wide open West they bring in the Internet and it goes to the LAN port on there now I labeled it when I chose one of the network interfaces as when I chose the other one as LAN when I set up XenServer so when I loaded it the first interface it found I set to be the management port to manage and and this to be like basically a second network card inside the machine and the only thing attached to that network card is the LAN side of pfSense that's it so in in worries of if someone was on the internet could they get to the management side no it's physically not connected is connected to one network interface card and at network interface card is then mapped to the pfSense machine from there pfSense hands out dhcp addresses to the land work now the land network is dual you have the internal sides and the external side is physical one but it goes in and out the same way so you attach things to that same network interface the plex server I have the backup server ends an orchestra all on this land part and then externally just a standard dumb switch no smart switch no manage switch goes out of there goes to my unify a PLR and there's my chromecast laptops phones there's a couple chromecast's out there these are the wireless devices and of course the kids kept phones and I didn't bother duplicating but there's tablets as well or any Internet device we don't have any IOT devices in the house so it really is just laptops and tablets and chromecast's and things connected to it for the wireless Andrew and friends come over now the things that are hardwired are the gaming system always hardware your gaming system best way to do it so if you're having lag issues it's not because of interference so the gaming system we have which is also the only Windows machine I own be well Linux gaming whole other topic it runs a lot of games doesn't run all the games so the gaming system is hardwired in and I have the chromecast ultra which supports network check and that's because of 4k content it works really good and I may have noticed I have an APL R that does not support AC I'm just not worried about that extra speed going out but it does cause some hiccups occasionally if you were trying to stream 4k content to a chromecast they recommend a faster connection I like a hard-line connection because it is a much more immediate connection I've tried both and each chromecast none of these are connected to 4k TVs matter of fact one of them's an older TV my son has 720 so it's just not it doesn't matter works great so there's to chromecast there one chromecast or big TV in our living room and that's the chromecast ultra and then the servers running inside here the backup server just runs sync thing and syncs all the data from my network at my office over to my house for a backup now being that is business data and things like that one we encrypt before it leaves second when we landed on here it's also encrypted on a hard drive so if you were to physically steal my XenServer when these boot up there using Lux encryption and you have to manually type in the password now this is done inside of the virtual machine has nothing to do is then at all it's the West my debian machines now that being said we'll show the usage on it it's very minimal amount of RAM and processor power at all it takes and this actually allows for hourly syncing of our critical data to this via syncing via VPN so everything's locked down and secure and that's really all this box does but of course you know something could happen someone could take something which is why I use encryption now we encrypt it before it leaves we actually the way we do the encryption for our database backups we encrypt it before it ever gets off of the servers that run the database which they're locked down they need their locks encryption as well just like the FreeNAS has its own encryption I believes GLI encryption we use that as well so everything top to bottom security is really really at the forefront of all this so even if you were able to get the password of this you would only find more encrypted data I just wanted to cover that a lot of people ask me about that and I've talked about it and maybe it needs its own video just talking about top to bottom you know encrypting each step of the way so that being said it doesn't use a whole lot of horsepower it runs sync thing did a whole video on singing I'm still using it I think that was like a year or two ago I did the video it still hums along really well and any what it does is you take two folders you point them at each other picture a self-hosted version a Dropbox if you're not familiar thing thing but I'll leave a link in a video leaving in the description so you can go watch the same thing video if you want to understand it but that's how all the data gets over to here and it's really lightweight because this only has 512 Ram assigned to it I think it's using not even a quarter that I think it uses a hundred Meg's of memory less Megan that gig 100 Meg's half a gig of ram assigned to it now the Plex here is a little bit different story I found with some of the larger content once you get over and this is that once again running Debian Plex seems to want a getting a ram for 1080p 60 frames a second content it has a little bit of a struggle with that processor was some and it comes down to how it's encoded 4k content so I have two gigs assigned to it and it does work depending how it's encoded it'll play 4k but it has no problem putting any 1080 content that work she works great no that barely even sweats the processor but it is a substantial four times more data when you're trying to run 4k I don't really do much for K content matter of fact I don't really do much what Plex says he have a lot of storage because there's just a few things on there and of course that Orchestra I like Zen Orchestra because it gives me a nice web interface to manage all of this now important side note here when you virtualize pfsense and PF sense is your DHCP server because my internet provider is in a bridge mode it provides nothing it bridges it over and gives me one public IP address which you obviously don't want to assign to the LAN so one of these I'm going to cover is how we have the startups working because if the XenServer reboots the XenServer has a static IP so that's out of the way but if you have other devices in your network that are not don't have static IP so they have no idea how to work at all and I actually let pfsense hand out all the DCP addresses so I have it set up to boot up pfsense in the case of power failure unattended this is not a checkbox invent server it's adding an RC local from the command line to it with a script that I will also leave in the description below so it's an open source script believe I got it off github I'll leave the github link if I get it off github if not I'll just I'll show you the script it's really simple but you have a startup script that kicks off to automatically start the boxes upon power failure of Zen the Zen will do this in a clustered mode but the default action is not doing it so that's how this is set up so on reboot it will automatically bring up pfSense again and you have to set a delay in there and I'm going to show you what that is when we get to that part so that's a pretty basics of it now we're actually gonna show you what the system looks like and the one thing you have to do to P of sense that one thing you should know don't don't click away folks of how to set it up it's the way you set up the networking it so let's jump over to PF sense now all right so the one trick that I really had to do here the problem I ran into and had to fix was the network performance was terrible and this was I mean I think I mentioned this in my virtualization lab tutorial but I'm gonna reiterate it again and this may be different for different things or if I loaded the drivers for Zen it would fix this now what happens is pfSense wants to talk directly to the hardware that's fine this is virtual hardware so being virtual hardware we had to check a couple boxes here and you can see just make it look like this the hardware checks some offloading Hardware TCP segment offloading and hardware large receive offloading is all set there now the other instance where you run into this problem is going to be if you haven't incompatible or not fully compatible I should say network card with PF sense you will have to check these boxes because it says that like for example with some real tech cards as noted in here either and what it does is is a hard low hardware offloading some of the processing the problem is because it's a emulated network adapter this is the only trouble I've had at all running this in here now that being said this pfsense is not super memory intensive I only have one and a half gigs of memory because there's only eight gigs total install virtual server so I got one-and-a-half gigs assigned to pfsense and this is running and I'm gonna be doing a video on this soon this is part of the reason I'm doing it the house to this is running snort it's running what else we have in here we get the snort we have the traffic shaping with the start of the PF blocker engies in here I've been doing a lot of testing and I might be testing at a home so I can get you guys some more data of like home use and things like that because there's so much more going on in our network to play with it and I'm less worried about accidentally screwing something up at home because it won't show my business down so I've got some more videos that are up and coming on this but all these things run perfectly fine we'll only a couple processors assigned twelve and now I wanted to jump over to the system itself on Zen I'm gonna use screen connect to remote into my computer at home to show the Zen running directly on there and if you're wondering what you're seeing here this is the remote support tool we use called screen connect and what screen connect does is allows us to remote into machines and it's convenient so I'm using it for my machine now - now the little of case you see here is because I did not update this a 7.3 version of Zen and I'll talk about that well real briefly here you probably seen a video I did complaining about some of the things and features they took away from it it's also tricky because I can't load it on here because it needs internet access to load and it wants all the virtual machines shut down so I'm actually if I do decide to load it I have to copy it onto like a thumb drive and install it because it won't have it has to have no virtual machines running before it'll start the download so that is at least one hiccup I've run into because once you shut down pfSense I have no internet access so let's take a look here at the networking we'll start there I do have this sign here but I'm actually not using it right now Tom's LAN of Zen where you can create internal ones I got a whole tour of that on my virtualization of how you can create internal only networks but I'm actually not using any data for the virtual machines here is Nick zero and Nick one and I added a description to cable modem so the only thing the only virtual machine connected to Nick one is pfsense it has two network interfaces assigned to it so if we look over here at the pfsense machine we go to networking we see two interfaces LAN LAN now when like I said it's the one that connects to the cable modem so the other virtual machines let's go to my backup server only have one it's attached tool and here's my plex server go to networking and it's attached tool and so each one of these is only attached to where they're supposed to be attached now let's get into details I think I can zoom in a little bit here because I want to talk about the performance we're gonna jump over to here and this machine's been in for about a month for me zoom out maybe just a little bit so now we're looking at some of the performance this machine's been on and I got it zoomed to your month so you can see January back to December it was right around just about 30 days ago that I loaded this so here's where the peak is for the rights back and forth to the hard drives where I set up all the virtual machines unloaded them after that not much at all because it just doesn't take a lot of power to run these and that was kind of a nice thing because they had these separate machines and obviously my closet like was warm all the time where I kept them and the power bill would suffer from it so being able to consolidate everything down to one machine makes it really easy now well the machine I didn't mention by just did the video on I set up the piehole at home so me scoot this over a little bit and that's the one extra server you see here is the piehole server and it's another example why I like this having this virtual server at home I just can spend something up my kids help are guinea pigs for testing how good the pie hole was working in our like a real world uses we used it for a couple weeks and that's how I generated the data for that video but once again will you look at the performance specifically on that it takes nothing matter of fact only had 512 assigned to it now let's look at some of the individual servers here's the backup server like samba it's actively backing up right now and let's zoom into like the one hour because we do hourly backups here's those little bumps from the hourly backup it almost uses can't even tell but it doesn't look like maybe a couple percent the CPU power you know you look yeah six point three percent is what we're hitting to backup that nurse about I don't know quite a few gigs worth of files I don't have the exact number but it syncs some it pulls it through our databases aren't that big so it's not not too much consolidate it's just important stuff the performance of here running all the things that I'm running the only thing that might slow it down if you're downloading when I download like a Linux just show at home and I'm using a torrent I'll watch some spikes on here you'll see little Peaks here and there and that's where I did that where you see the network performance once again though with 4 CPUs assigned to see pfSense nothing is fully saturating it or using full performance now the only exception to the hats going to be when here is the you see a lot more usage on this system here and I was playing around with it trying to play some 4k content it Peaks it out right away but 1080 play is perfectly fine and the Tom's house server mostly is used for plex there's a few miscellaneous things I have on there but is mostly used for Plex and works wonderful for it so 1080 content doesn't stress it plays great no no issues at all on there now the important part though something we want to talk about here is go in here we're gonna go over to the council I'll open it up in putty alright here's that really simple script and like I said I'll make sure there's a link to this below or link to where it was or you can get you a copy of this script really simple you put the names in here so it says names equal and the Duke who I care about is pfSense and xoa and VMs is play log file matches UID you give the UI these herbs and machine names which is easy enough to get let me switch over to that screen if you look at any virtual machine there's the ID for each one so you take the UUID of the virtual machine and you can actually just copy it and there it is and then you tell it to Auto starts VMs and that's all this scripturally does it's pretty straightforward nothing real exciting about it it has a VM a starter blog it throws this information here Auto starts VMs but where do you put it so the way I set it up was and we'll just set they go it it's in oops so you go over to your two Etsy /r CD and if I do an LS here I created this file December 9th this when I set this up our C dot local and all that file does we're going to go ahead and get this is it's a generic RC local file I had an ad sleep 360 because it would in it RC local a little bit too early sometimes before the storage is ready so 360 seconds is leni of time for the machine to settle down be completely happy after a rebooting place anything went wrong you know doing a disk check and things like that then it just goes and runs start vmstat SH that's it so this file in our see local boom you can do this now I found this in the Zen forums this is not like unheard of knowledge so speak I didn't invent this at all and it's really simple and it even tells you what to do here it says make sure you run chmod plus X because first thing Tom didn't do is read this and then forgot to set it executable just an oversight on my part but it works perfectly fine I have pulled the plug on the machine a couple times to make sure it will completely come up and that's great other side of it though when you're running updates and you want to run the update reboot and run another update you forget and then the machine starts in the middle of you trying to run an update and get other errors so think about this if you're running updates make sure it's in maintenance mode so these won't the machines won't start when you do that but other than that it's not been a problem at all for doing the things that I do on at home and having a Home Server it's been really handy virtualizing pfSense has been fun because I can do custom things and you know before I make any crazy changes when I'm turning things on I just ride and make a snapshot real quick and go ok I'm gonna change this or change that and you make a snapshot of it make those weird changes or tests that lead to more videos and it gives me a better than setting up a more sterile lab environment here I get a whole environment of what in the world are my kids gonna load on their laptop and what is it all gonna do and everything else so it's kind of it's kind of fun being able do this at the home level so all my home servers and are easy to access the other reason I use XenServer and I got update this one too the same version I have here but it should work perfectly fine for importing VMs is in the event something absolutely horrible were to happen to my building to my business or anything like that I want to make sure I have a Zen server at the ready so I have a spare XenServer here so we have two of them one's not even plugged in it's just I turn it on I load the updates and I keep it turned off it has its own local storage because I constantly backup every weekend with all my backups I have always recent copies of everything so if I need to restore a server I can restore a server so if something happens to the building something tragic a flood a fire whatever the unexpected I keep extra copies of my servers they take them each weekend home on an encrypted drive so I can really quickly restore and if I had to run my business from my house with my Zen server I could I could restore you know some of the critical things we may need and make things happen like even we've self host even our remote support tools so we would just change the DNS entries to point to my house when we would grab our laptops around the table and plan this is as part of the disaster recovery planning is having a plan B for this of course because I'm exporting them in are in the standard format which is something I like about Zen I could upload them to a cloud hosted provider as well and not have them in-house so there's other plans other options but that's it for running a virtual server and at my house and how I got it set up if you like to count here like to describe if you have questions about this project let me know leave them in the comments below or join our forums where I can hopefully be more interactive and answer questions about it alright thanks

Channel: Lawrence Systems

Views: 126,680

Keywords: Citrix XenServer, Home Virtualization, citrix, xenserver, virtualization, tutorial, linux, xen, xenserver 7, hypervisor, xen server, technology, virtual pfsense, pfsense, router, firewall, virtualbox, pfsense tutorial, pfsense (software), server, home lab, virtual

Id: HUzWnkfxcok

Length: 23min 23sec (1403 seconds)

Published: Sun Jan 14 2018