Migrating my PERSONAL SERVER from TrueNAS to Proxmox + Cockpit

so the big nightmare behind me is open because today I am going to shuffle some servers around this is my little Terra Master Nas Terra Master sent it to me I talked about it in a previous video link up here and this is currently doing basically nothing and I'm going to change that guess I should back up a bit I currently have 10 servers in the house doing server things I'd like to reduce the number down to seven first up I got big store I've made a video about him before link up here to that video this is my proxbox backup server it has my Force ass drives throws my backups also has my tape video up here about the lto tape Adventure this one's just going into the closet it's running fine I love it next up we have mega lab which is my Open Bench test here and it's also doing fine so we're going to leave that there and that is running proximox8 and it is doing proximox stuff this the Terra Master Nas is also running proximox8 and I've been using it basically just to film videos so I made a video about setting it up it has a 500 gig nvme drive in it in two two terabyte WD red drives we're trying to find I've been using it as sort of a test bed but I really only use it when I'm making a video about running software on proxmox and I can do that in any of my proximatic systems the only reason I use this one is because it doesn't have much on it um so this is basically doing nothing for me right now it's in the closet behind me I've got my open sense firewall that's going to stay nothing wrong with that also in the closet behind me I have my three node cluster video up here on that one that's the one I've used to turn into my proximox cluster test it's all the one I test Seth with and I actually have a virtualized five node cluster running on Mega lab as well that I can test cluster stuff on too so that one's currently powered off but I have a Seth project coming up so that's going to get power back on soon also in closet behind me I have a node called minilab minilab is my first Home Server well that's right my second it's my first proxbox server so it's a well you'll see in a few minutes but basically that runs home assistant frigate my mqtt broker influxdb and it has been running it for a very long time so it has like 80 gigs of data on in flexdb which is an insane amount of data just to be collecting from home assistant also chilling in the closet I have my rack mount server there's another video up here on that one that one is called iridium that is my it's an AMD construction equipment CPU Apu it's my true Nest server so anyway I'm kind of upset with trunass for being true Nas as usual um I want to get off of true Nas and on to proxmox for my storage I made a video another link up here in the cards on setting up a storage server in proxmox it's a pretty good Nas video and I'd like to use that and at the same time I want to move my semi-home assistant stuff from home assistant OS onto home assistant core which is the python base instead of the OS based docker-based system so I can have more control of it because I don't really use add-ons anyway I basically do everything externally so I don't really have a need for the home assistant OS and getting rid of that we'll get rid of the VM layer that I have running for that I'm also doing some testing of frigate on the zema board now that's the 10th computer is azima board and so frigate will eventually move on to the system as well so basically I'm going to consolidate minilab this and iridium the truenast server just to be on this so this is a really low power Nas box it's got two two and a half gig ports two USB 3 HDMI and DC jack and then internally it's got two nvme drives both with four lane pcie Lanes to the CPU and it's got two SATA Bays so basically what I'm going to do is I'm going to have a I have a 500 gig invade me drive in here and it's going to stay in the second nvme slot I'm going to put my Coral TPU which is for frigate and in these two drives here I'm going to put the two six terabyte drives that I have in iridium the file server and to avoid disruption to people I'm gonna do this in two steps first I'm going to take down home assistant which will make everyone unhappy because all the lights will stop working they'll stop being automated they won't stop working and if they're not automated they're basically don't exist and people won't use them just live in the dark so those are going to turn off for a little bit I'm going to migrate that over because I need to take the coral TPU out of that node and put into this one then I'm going to boot this one up and migrate the VMS over I'm just going to restore backup from PBS so I'm going to do a backup shut it down then restore the backup onto this node once that's good and running again then I'm going to shut down iridium because that interrupts video production when I have video my video storage move the take the physical drives out of truenas and put them in here they're already ZFS formatted I can just bring in the pool and share it so that's basically how today is going to go and uh let's see if that ends up happening or not this video is sponsored by protoark and their xkm01 foldable full-size keyboard and mouse are you a CIS admin on the go who's sick and tired of carrying around a USB keyboard and mouse to work on computers with a Proto Arc xkm01 you can whip out the handy carrying case pop out the included USB dongle and unfold the full-size wireless keyboard and mouse for those emergency server operations or maybe like me you routinely rebuild your computers for fun the xkm01 can fold out of the way when not in use and supports both a USB dongle as well as two Bluetooth profiles easily switched for both the keyboard and the mouse this is great for reducing clutter on my workbench both the keyboard and mouse can be charged over USBC just like all of your modern gadgets and the mouse includes a pocket for the dongle as well the included carrying case fits the keyboard and mouse comfortably in my backpack ready to help in whatever tech support Quest I have to deal with today check out my link in the video description to get yours today so everyone this is minilab my first ever proxmox server it is an ASRock a300 desk mini with an AMD ryzen 2400 G Raven Ridge APU some of you may say what is the a300 chipset I thought AMD only made a 300 b350 and was the other one x370 and x399 a300 is not actually a chipset it's a chipset what it basically means is we take all the i o on the Zen CPU die and we run it out to the connector as is with no chipset so the Zen die has some SATA some pcie um some USB 3 and basically oh and three display ports so we take DisplayPort we can DisplayPort HDMI it has a built-in display port to VGA converter chip on board that's not part of the chipset though there's a real tech neck that's on a pcie lane and there are USB 3 ports and that's it internally there is an m.2 Wi-Fi slot an m.2 nvme slot and two SATA ports and all of them go straight to the CPU so essentially this is a Zen CPU broken out into a little circuit board and I bought this thing many years ago when I was planning on 3D printing my own computer because the motherboard is so small that it'll fit within the print volume of my prusa I3 this is back before I had the bigger printer and an ITX board was not an option for the bigger printer so I bought this and I was going to have it down here and my plan was to build an Electronics Workshop down here and I was going to have a monitor for looking up schematics and doing internet and stuff this is going to be the computer I was going to 3D print with it and quickly this turned into a server as does everything else in my life so uh yeah it has been decommissioned I already did backups from here onto the backup server I flagged them all what's the flag called throw flag that's protected so the backup server won't clean them up automatically I gave them all a name let's say they were for migration so This Server is decommissioned I don't have to do any more to it so now comes in Tara so all I'm going to do to Tara is take it apart and put in one of the two Coral tpus so this has two Coral tpus in it one is a m.2 Wi-Fi and one is an m.2 SSD key so I'm going to take the SSD key one out and put it into Terra and I'm going to take the Wi-Fi one out and I'm going to put it upstairs in the zema board that I'm using to test forget so let's get started so let's get the coral tpus out of here this is like the easiest PC case ever to take apart there's four screws and the motherboard slides out I'm gonna get these four screws and there's a little connector for the power switch nice so this is the coral on the m.2 carrier in the SSD key and this is the Dual Coral in the Wi-Fi key the Dual Coral is a bit weird it's so the Wi-Fi key has a number of i o on it including USB sdio and two separate pcie by one Lanes so the coral TPU has two separate tpus Each of which is a pcie by one but most boards only wire one of the two pcie lanes it's not that it's a two lane piece I.E it's two separate pcie by ones and this board only has one of them wired so only one of the tpus works which means I have two tpus instead of three but I was very aware that it probably wouldn't work and it cost about the same price and the Wi-Fi key was one of the only ones I could buy so it is what it is just going to keep this screw here for safe keeping up you out we're gonna need you to put your screw back in for safekeeping I love how this has just a big front panel i o connector here it doesn't have like the individual little wires you get on Modern PCS let's just slides how does it slide look at that slide back in nice so this is officially retired I can now use it for other projects so you may recall taking this apart it's pretty easy I'm going to pull the two drives out first I'm actually going to keep the drives out because I'm going to swap from the two terabyte WD Reds to the six terabytes that I already have in iridium but I'm going to do that after I get home assistant up and running again migrating one server at a time here nope come on this group feels a little bit stripped come on there we go okay and oh ah just trip one anyway I don't need that one okay so here we've got the 960 Evo and my eight gig extra Ram there's four gigs of RAM on the back so 12 gigs so this system has 12 gigs of RAM so I'm going down from 16 to 12 and I'm also merging from 216 gig systems ZFS true now system and the proximox system but at the same time I'm migrating everything to Containers so I'm not going to have my frigate VM and I'm not going to have my home assistant VM they're going to be containers now in the new setup so I got my 960 this is the 500 gig drive this is an upgrade to the other system only had a 240 gig Drive it was only SATA not ndme so here's the oh I need that screw here's the other slot and here is my Coral let's get you back home [Music] I'm gonna plug the fans back in okay so this is set up now I got two two and a half gig Lan and a USB 3. so another reason I'm much more open to switching to the Terra Master now is because I got a new desktop another video up here in the cards and I'm loading a two and a half gig there so previously I had five gig up there and 10 gig down here to the Iridium server so if I connect this with two and a half gig to the switch then I have two and a half gig from here to there and from there to my desktop so I still maintain two and a half gig path for video files I'm going to hook the second two and a half gig up to the private land that's used by the ceph cluster so it can do storage and stuff so then I can use this as like bulk storage of whatever I need when I'm doing SEF testing and then I'm going to use USB 3 with that stupid cable I bought to make a short this thing I bought this USB 3 data transfer cable and it really kind of sucks you barely get over a gigabit with it but I'm going to use that to connect from here to Big Store direct so all of my proximox backups will go direct to big store and they'll cut out the switch entirely which means that doesn't mean anything for this system because this system wouldn't care but it means on a big store side it has two gigabit paths one that's gigabit to the switch and one is gigabit to this so since a lot of the backups are coming from the Terra master and some of them are coming from Mega Lab at the same time it'll split bandwidth for backups to big store so that's what's going on with this setup once I get it set up and yeah it's going to boot it up and see what it looks like so it's a new day I got the new server set up well newest server server new role setup I also took the backup server and I moved it into the closet so it's chilling in there happily so you got room to work today basically I just restored the backups from PBS onto the proxbox system and started them and that was it nothing else to talk about it just worked all the settings imported they have the same Mac address because it's part of the config file you can choose to import them to a new number and it'll by default pick a new number for them but I manually set them back to the number they used to be so in proximox backup server they'll still retain the same backup history and everything um so yeah everything's working again I did testing to make sure all of my Z-Wave and zigbee devices are working again and they are doing great and everything of mqtt is doing fine the stuff that needed node red so my old home assistant stuff I started with node red before I was in a home assistant and that's all working too so it just worked so now it's time to move the file server so last night I had the file server kick off a backup to the proximox backup server and that ran and then this morning just before I shut it down I had it kick off a backup to itself so there's two six terabyte drives and one five terabyte drive and before I had a proxbox backup server and I was doing good backups I would back up the video data from the six terabyte pool to the five terabyte drive just as a copy operation and I did that so I'm going to keep the five terabyte drive separate I'm not going to mess with it so it'll be another backup of anything that's changed in the last couple of hours which shouldn't be anything I don't I didn't upload any video files or anything to be minor changes that I'm not sure what they are but it's possible so let's get it out of the rack and get started it's getting the server out of the rack is always a pain because the direct attached copper cable for 10 gig interacts with the pcie slot bracket so it can't come out so when I tug on this retention clip it gets stuck on the slot cover for that pcie slot which is really unfortunate so I can't get it out without taking the card out of the machine so I had to pull the whole thing out with this two meter DAC cable attached I got the power cable too but that one wasn't attached so let's take this apart and see what's inside okay so I made a video about the system a long time ago and there's probably a link to it somewhere or maybe there was earlier in the video but basically this is my first true Nas system so I built this as soon as I got out of college so it's got an AMD excavator pile drive or something like that it's a four core four thread CPU it's got a full-size pcix 16 slot which I think is running it by two maybe it's running it by gen 3x8 or something like that I don't remember exactly that's what I got my 10 gig card in so this is an Intel X 520. uh 16 gigs of RAM I think yeah 16 gigs DDR3 Pico PSU these two are six terabyte drives and a ZFS mirror this is the boot drive because truenas refuses to boot off your data drives and then a five terabyte desktop drive and this is my this was my backup before I got the backup server so my goal today is to take these two hard drives out and put them in these hard drive cages to put them into the Terra server then I'm going to import the ZFS pool as is and go from there foreign Drive trays from Terra Master are really easy just pop this like pin piece out both sides and it just comes right off [Music] these drives are manufactured in February of 2018 and I bought them not that long after that I bought them in summer of 2018. so the system wasn't on full time in 2018 and 2019 but in 2020 it got set up as this and it's been running that way ever since so these have had a lot of power on hours they haven't had a lot of power Cycles because they run full time fun fact the 10 terabyte SAS drives that I bought for the backup server they have less power on hours than these and I bought those used so some company had retired them after three years of power on hours and these are getting close to three and a half to maybe even four years so it's time to replace these six terabyte drives at least for critical stuff so my plan is well I was hoping to migrate to ceph sooner rather than later I'm not sure if that's going to end up happening sooner or if that'll end up happening later but in any case what I'm probably going to do is buy a pair of something between 10 and 16 terabyte drives and do the ZFS mirror replace method where I take OneDrive out and replace with another one re-silver okay take the other drive out and replace re-silver and then I end up with a mirror repair that's been upgraded so that's my plan for these six terabytes they're going in for now I'm going to upgrade relatively soon either to safer to some bigger guys you ever watch your own YouTube video to remember how to do something I'm installing cockpit the file server for my pretty good Nas video I'm watching my own video remember how to do it so now that the system is up and running Sherlock and I are going to give you an overview of what it looks like so here's the system CPU usage is higher than I would like I'm not entirely sure what's using all of that now usage looks about right through ZFS so I'm not worried about that one this VM and these two containers are what I had on the old system and then iridium is the new container that replaced the old iridium trunass server so a bit of History I started using free Nas back in 2018 on the Iridium system and I was quickly kind of unhappy with it I used BSD jails for a while but it was hard to find software that ran wheel and BSD compared to Linux so I switched to Ubuntu 18 and I ran lxc containers manually and Samba manually and so some of these lxc containers have been moved into proximox since then so Telstar is one of them that was my first home automation system I based it around node red and mqtt later on I had influx DB and other stuff to it but originally it was just going to be mqtt based and mqtt is like stateless message passing protocol so I named it after this um so this got backed up from the original system got imported into proximox has been running ever since I also have a AP controller that's from my wireless APS from tp-linkomata I have the older software because I have older APS that aren't supported by the newer ones because they don't do great with backwards compatibility tp-link even if their current Hardware stuff is pretty good mirror that is the home assistant server so home assistant obviously home assistant OS is something I've used for a long a couple years now and going to migrate that eventually to be a container as well because I don't really need the additional add-on features of the docker system that they have but for now it's still a VM and I'm going to keep it that way and last but not least iridium so iridium used to be at the physical server and for reasons I didn't want to change the name because it's connected to by so many things around the house that connect to it over Samba so I took the old DNS name for iridium and I moved it to the new container in the same IP address so the container now fills the role that the server used to fill all the shares should be the same you just have to reconnect same users passwords everything so essentially followed my video on installing cockpit on daily in 12 instead of 11 and that worked pretty well I did a couple things that I'll show here so here's cockpit it's running so I have file sharing set up in my case I I enabled Mac OS access which turned on fruit and set some more fruit settings to make Mac happy and I had to do this to make my iPhone happy because part of my workflow is to upload files from my iPhone directly to the file server I also set Shadow copy turned on and so I have this set to use the format that sanoid uses so if I pop up with a view here so this is project folder if I look at this in Windows previous versions you can see there's three versions of it every half hour that one was a mistake um to do one of the complications of the unprivileged container that I didn't realize would be such a Troublesome thing when I made the previous video it was uh uid and GID mapping so unprivileged containers run at 100 000 and dealing with uids that are shared between containers can be challenging so I didn't want to just make it on pre or just make it privileged so I followed this tutorial on proxmox to idmap in my case at idmapped ID 1000 which is user projects and all the systems so that I basically just have to ID map on the ad group so the the samba users all have to be member of group 1000 and that'll give them permissions on the host system uid gid1000 which I can then set in CFS so my data sets for bulk data my video projects personal data that is all outside of the proximox system and it's in separate ZFS data sets so I'm essentially doing ZFS management manually for those and backups and things like that using proc box backup client so I'm using sanoi to take ZFS snapshots and it's taking them every half hour so there's different configurations you can do but I told to keep the last 144 half hour snapshots that turns out to three days so in Shadow copy and windows you get the last three days as 144 snapshots which is an outrageous number so if you delete a file on accident you'll probably be able to find it I'm also keeping the last month's worth of dailies and then three months worth of monthlies so that's a lot of snapshots another thing that's come up is my power usage has gone down as a result of this so there's two different Power circuits that power that area downstairs so if I look here this is shop lights this is what is in the room with the storage closet so it used to be at about 205 Watts or so but it could Peak up to 210 or maybe more than that if it was doing heavy work so now today after all is said and done these big flickers here are actually the lights themselves now we're running about 214 213 so we ended up 10 or 15 watts to this circuit now the shop plug circuit is a different one this is where the backups were used to be so the backup server here was pulling 90 Watts and then these spikes or other things happening so now backup server moved from the shop plug circuit to the shop light circuit our power went down to 33 Watts so I took away 60 watts here and I moved things around here so the backup server got added the Iridium server got deleted so all in all I'm saving about 50 Watts and civility watts is not nothing I'm pretty happy about that so finally now that I'm done moving everything over to the new server I'm going to take the server apart fully scavenge all the parts lay them all out so I can build new projects out of them all my data has been migrated so I don't need the backup copies anymore well that's not true the Z Pool moved to the new system so that copy is good the backup was always good because in the backup server and I just did a tape backup like two days ago so I have a two day old tape backup which is perfect so the backup hard drive that used to be my only backup that's now been duplicated by the backup server and by tapes is obsolete so I can take the five terabyte drive out I got a Ziploc to hold all the screws because little screws are such a pain so this thing is held it up yeah it was held in with double-sided tape I guess not anymore so all of these parts are going to go into other servers some days I build them so this Intel x520 I have two of them now so that's two 10 gig cards and I have a 25 gig card so that would be enough to build a full three node cluster ideally I could put this in the backup server but the backup server doesn't have enough slots for that this hard drive is actually still in part of its case because you can see I shucked it from a by USB chassis maybe I should put it back in the USB chassis that would be useful but uh maybe not I was hoping to use this drive for security cameras because that's kind of stuff that doesn't need backups if I have an important event happen on security camera I'll export that as a single file and save that but in General Security data is basically worthless once it's created at least that's my view on it so this for those of you who haven't seen it is a Pico PSU so it is an ATX connector with a power supply mounted it takes a 12 volt input from a wall brick and feeds that straight on through to everything takes 12 volts so The Regulators on board only do 5 volt 3.3 volt negative voltages so this one I think can do 120 watts so I'm going to move this over to the mega lab the mega lab is a 65 watt TDP system or CPU the only thing attached is the RAM and SSD and the 25 gig card so it's only running a 10 gig with one lane so the reason for that is because the mega lab has a 500 watt sfx power supply and I can use that to test the uh well you'll see in a sec so this is going to go to that computer that's where it's destined I can take off all these extra Molex expanders because I won't need them anymore I had to have a bunch of them to get all the Theta power I needed case had two fans in it and they were super super loud so after I bought this case I replaced one of them with a knock to a fan and unplugged the other one so this little board the uh what is it a68 and 5600 so this is a I have no idea what model it is but it's a AMD excavator processor it's an ITX board I'm going to try to use this to test the Xeon fee this has iomu support it's about 10 years old of a processor design and znfe is about that old too so I'm hoping that they go well together if not I can always test Mega lab but for the Zeon fee I really need a like dedicated system because I'm gonna have to be installing old versions of Centos because they're designed for red hat design for red hat or Suzy so my options are pretty much to run old sentos and um yeah so this has the pcie slot I need for the Xeon fee so for those of you who haven't seen this is my Intel Zeon fee it's a coprocessor this is derived from the early Intel larabee GPU prototypes from about like 2009 or something like that so this has a hell of a lot of cores it does four-way smt so four-way multi-threading and it has 60 it's just a couple less than 64. it's an odd number but that's how many full cores it has and each one does four ways so that's like 200 and 144 threads or something stupid like that this had a precursor to AVX 512 that is not the same and is not supported by GCC it was only supported by GCC so they could build the Linux kernel for it because it runs Linux itself so I'm hoping to get this thing running I don't have any fans big enough I bought some fans they're not big enough so I haven't had any hardware to do a full test I was getting in touch with my threadripper but that died so we're gonna see if this little AMD system is enough to just power on and Host this because I don't expect that to do much processing but as long as it has the iOS IO mmu features I need the Xeon fee should work I got a 500 watt power supply hopefully that's big enough too because this Beast needs a ton of power so I got a pcie eight pin and a pcie six pin and apparently this thing needs like 25 CFM of airflow which is crazy um so I need some big big blower fans but yeah this is going to be a future project I'm hoping to get it to transcode prores because in theory um Hardware Hardware accelerated codecs like h264 av1 are really really efficient if they work if um if they're not Hardware accelerated though you end up with some really high CPU load trying to encode and decode them and if I do stuff in prores then I'm probably going to want some sort of way to transcode all my files into prores and how to prores that's not my desktop computer whirling away at it so that's what I'm going to try to get this to do I know there's a bill of ffmpeg that runs on this I don't know how well it runs so this is upcoming long term not near term I've resolved I've extracted all this Hardware from the old server so at this point in the video I'm down to eight servers running I still have the Zima board upstairs running frigate and that's going to keep running for a good till I'm done doing testing of different ways of doing AI detection so the TPU the CPU detector open vinyl detector and I'm trying to Benchmark those to get some data on them before I move frigate down to the server but it's eventually going to run on the server as well and by then I'll be down to seven servers so aren't you proud of me for uh slimming down my home up a bit so yeah as always if you guys want to see more for future videos don't forget to like And subscribe hopefully I can get around to some of these projects soon but the timetable is always variable um message me on Discord I have a Discord server linked down below if you're interested discord's a great place to chat with me about all kinds of weird topics what else oh I have a Kofi if you're interested in tipping me it's not like a monthly donation thing or anything like patreon it's just one time and that's open if you're interested and as always I'll see you on the next adventure

Channel: apalrd's adventures

Views: 22,360

Keywords:

Id: _D6VVlqWwSA

Length: 33min 30sec (2010 seconds)

Published: Wed Sep 20 2023