Are your utility bills driving you into debt? Is your mum kicking you out of the basement
because your server consumes more than all Chinese mining farms combined? Is Radiation Safety Commitee knocking at your
door because your home server looks like a juicy
piece of enriched plutonium from space? Well, fret not, because in this video I’ll
show you how to build a super power efficient home server/NAS, no matter your use case. Whether you need a low power mini PC or a
beefy Proxmox host I’m gonna tell you how to get the best performance
per watt and make sure your homelab hobby doesn’t
stick out like a sore thumb on your power bills. So strap in, and let’s get started. This is my current home server. I primarily use it for video editing over
the 10 gig connection and media storage. It draws around 23 watts from the wall on
average, and with current energy prices in Germany
this machine costs me 5 euros and 37 cents per month to run 24/7. Now, considering the fact that a lot of r/homelab
users run machines that draw 120+ watts at idle,
I think 23 watts is pretty good. Keep in mind that this isn’t an Intel NUC
or a similar small form factor compuer I’m running a full-blown rackmount server
with four 3.5 inch hard drives, two NVMe drives and a 10 gigabit networking
card. So, why would you even spend time optimizing the power consumption of your server in the first place? Well, If you live in a country with expensive
electricity, the difference between running a 50W home
server and running a 100W home server will definitely be noticeable on your utilities
bill. And considering the looming energy crisis
in Europe, this might become very relevant for a lot
of people in the near future. A power efficient server will also stay cool
and quiet so if you live in a country with hot climate
or don’t like the fan noise, this might also be the way to go. Whatever your personal reason is,
designing your home server to be as power efficient as possible
is a great way to save money And I don’t know, personally,
I kind of like the idea of squeezing the most out of my system
while making it consume as little juice as posisble. Some people might find my power efficiency
obsession stupid and pointless especially if you live in a country with cheap
electricity or just don’t care about your utility bill. Well, good news, I don’t force you to become a power saving
freak like me and you’re free to ignore my advice or skip
this video altogether. But if at least one video finds the information
in this video useful, that means my work hasn’t been in vain. Building a power efficient server begins with,
well, picking power efficient components. And that’s not as simple as
buying a low TDP CPU and pairing it with a server motherboard First things first, if you want to save on
your utility bill, don’t buy old hardware By old I mean pretty much anything older than
Haswell or the 4th series of Intel CPUs. Older systems might be cheaper, but your savings will probably go towards paying the increased utility bill Even though older CPUs might have the same
nominal TDP than the newer ones, they’re much less efficient at idle
and might not support the power efficient C-States that the newer CPUs support. At the same time, newer doesn’t necessarily
mean “more power efficient”, And recently we’ve seen the PC components
like CPUs and graphic cards Become more and more power hungry for the
sake of ‘performance’. Pretty much any Intel CPU that is 6th gen
or newer will be pretty efficient at idle, And even though the newer CPUs might be slightly
more efficient, You will have to shell out more money for
a newer motherboard and a more expensive CPU So if you’re okay with the performance of
something like 6th or 7th gen CPUs, don’t chase the newer 12th and 13th CPUs
purely because of the power efficiency But what about AMD? Well, when it comes to Intel vs. AMD, Intel
systems tend to win When it comes to budget-ish systems,
Since they tend to consume less than equivalent AMD Ryzen machines. Ryzen CPUs also still seem to suffer from
the idle freeze bug on Linux, Which can only be fixed reilably by disabling
the power efficient C-states, Which is kind of counter-intuitive That being said, you can still build a pretty
power efficient system with a Ryzen CPU, And some users report as little as 7W power
consumption with a 4350G. One more thing I want to mention is that the
TDP spec means absolutely nothing for the real-world
power consumption in a home server. TDP only describes the power consumption under
load, And in a lot of cases, even that figure doesn’t
match what you’d see in real-life use. Despite consuming upwards of 100W under load,
Many modern processors can still enter a power-efficient idle state
In which they sip less than 1W. That also applies to the T-Series Intel CPUs These are pretty much the same chips as the
non-T models, just capped to a smaller TDP. This has no effect on the idle power consumption,
so don’t pay more money for a T-series CPU because of the supposed power efficiency Since your server will most likely be idling
most of the time, at least compared to a desktop PC, the idle draw is exactly the figure we’re
interested in and it can vary wildly depending on your motherboard,
PCIe devices, power supply, and so on Unfortunately, very few manufacturers and
reviewers publish the idle power consumption figures,
but luckily for us, there’s a whole community focused on building
power efficient computers, and it can be found on this German forum called
Hardwareluxx The forum members even maintain a database
of the most power efficient builds Sorted by consumption Which can be a great
help if you’re looking for power efficient components So let’s dive in As you can see, the most power efficient systems,
ones that consume as little as 1 to 4 watts, are Intel NUCs, laptop motherboards and basically
ultra small form factor PCs. Obviously, those computers are power efficient
for a reason they don’t have the most beefy CPUs in them And the number of features and ports is also
very limited You’ll rarely find more than one SATA or
M.2 slot, and PCIe is also usually out of the question. That’s the price you’ll have to pay for
ultimate power efficiency, And if you’re okay with not having those
features Then these options could be for you These machines should still have plenty of
power for Virtualization, Docker, Kubernetes, Proxmox, Or even running a media server or a Home Assistant Instance But they’re probably not the best choice
for a NAS since they lack expansion If you want your home server to be a bit more
capable, The next option is a desktop CPU and a miniITX
motherboard MiniITX motherboards tend to have less ports
and features than their ATX counterparts, and because of that they usually consume less
power You might think that the difference between
a miniITX motherboard and a full ATX mobo with
the same CPU is negligible, Aaaand you’d be wrong Most modern CPUs are pretty efficient at idle
no matter the motherboard, contributing as little as 1W of the overall
power consumption. However, if a motherboard isn’t really optimized
for power efficiency or has a bunch of onboard devices like multiple
Ethernet ports, Audio, serial, iKVM module and so on
those devices can add as much as 10 to 15W to your power draw, and disabling
them in BIOS doesn’t always help. However, not all miniITX motherboards are
made equal, and some models just aren’t very well optimized
for power efficiency, These miniITX motherboards from ASrock and
Fujitsu on the other hand seem to be especially good when it comes to
power efficiency So if you’re okay with only having one PCIe
slot, a miniITX motherboard is definitely an option
that you should consider Some server and workstation motherboards also
support PCIe bifurcation Which lets you connect multiple devices to
one PCIe slot using a bifurcated riser like this one. However, a bifurcated riser can cost as much
(or even more) than the motherboard itself
so it’s only worth it if you want to build a small form factor server in the first place But what if you need multiple PCIe slots? I have a 10 gigabit card and an NVMe adapter
that I need to plug into my motherboard, and the Asrock mobo that I used before only
had one PCIe slot, and no NVMe. Well, let’s once again take a look at the
spreadsheet. As you might be able to tell,
the most efficient motherboards aside from miniITX
and ultra-small-form factor PCs are from… Fujitsu This build in particular is very impressive
– an Intel Core i7-9700K and a Fujitsu D3643-H
combo only pulls 5,8 watts from the wall I already had a CPU handy, so I started looking
into Skylake motherboards from Fujitsu,
and I found this D3402 motherboard on eBay. With 5 SATA ports and 2 full-size PCIe x16
slots, it fit my needs perfectly. Now if you don’t have any parts handy
and want to build a new PC from scratch, I would recommend going for a newer Fujitsu
motherboard that supports at least 8th generation Intel CPUs Or at least I would, if you could actually
buy them At the moment, both 8th and 9th gen Fujitsu
motherboards are out of stock on pretty much all online
marketplaces New and used. But when they do crop up,
they’re usually not super expensive, so keep your eyes peeled Meanwhile, you can take a look at these 6th
and 7th gen motherboards They seem to be easier to find on eBay and
local marketplaces, and are also very power efficient. After rebuilding my system and replacing the
MiniITX Asrock motherboard with the Fujitsu one, I booted the machine up, waited for the drives
the spin down, and looked at the power consumption It pretty much stayed the same, which was
kind of disappointing, but at the same time it didn’t increase,
despite me switching to a bigger motherboard. On the other hand, similar builds in the spreadsheet
pull less than 10W. So how is it that my system still pulls 20? Well, let me introduce you to Package C-States. Package C-States are power efficient semi-idle
states that the system can go into when there isn’t
much going on The system is still technically ‘awake’
in this state and can do background tasks, but consumes
much less power There are usually 8 to 10 Package C-States
on modern systems, and the bigger the number, the bigger the
power savings C1 corresponds to high system activity, whereas
C8 is kind of a daydream The easiest way to see which C-state your system is in Is by using an command-line utility called `powertop` You can install it using your favorite package
manager, and after doing that let’s open it and go
to the “Idle states” tab. Keep in mind that we’re looking at Package
C-States on the left, not the CPU C-States. As you can see, my system only goes down to
C3, which is not ideal. What could be the culprit? Well, either the OS and software configuration, or pretty much any connected device that might just have crappy firmware The former is very easy to verify by booting
up a fresh Ubuntu LiveUSB, installing powertop, executing `powertop --auto-tune`
and then running powertop again and taking a look at the measurements. If it stays the same, the issue is with the
hardware. In my case, the culprit was my 10 gig networking
card, Mellanox ConnectX-3 It’s pretty old and I bought it for cheap
And because of its age, it doesn’t really support
proper PCIe power management. As a result, with the card installed,
the system wouldn’t go into deep idle Unfortunately, the “cheapest” SFP+ card
that supports ASPM is Mellanox ConnectX-4, which starts at €170
on the used market Not really worth the investment Another common culprit are cheap NVMe drives The companies making budget drives don’t usually put a lot of money in firmware and power optimizations, so with a lot of them, you won’t get proper
ASPM support, once again resulting in no deep idle I got two cheap SSDs, one from Sabrent and
one from Crucial and the former does not support ASPM
and won’t let the system go lower than C3 As you can see, with the Mellanox card and
the SSD disconnected, the system now goes as low as C8, which results
in 8W idle, even with four hard drives connected to it
in standby mode. Obviously, at this point it’s not really
worth it, since I would like to keep my
10 gig networking and my SSD so what are some other options for reducing
power consumption? Well, the thing is, even with all the right
hardware and a platform that can go down to C8 idle, You probably won’t see the power consumption
go as low as 8W with a conventional ATX power
supply And that’s because even power supplies with
Gold or Platinum rating have relatively poor efficiency at lower loads Which is understandable
If you’re building a conventional desktop PC, You probably aren’t planning to just let
it sit at idle And do nothing the whole day If you turn it on, you probably want to game
on it, watch YouTube, do some work, and so on Because of that, a lot of people who run low-power
systems Use alternative power supplies called PicoPSUs
paired with laptop power bricks Those PSUs are much more efficient at lower
loads, and as an added bonus, they’re also completely
fanless Thanks to their size, they also work
very well in small form factor systems. Obviously, using a PicoPSU only makes sense if your idle power consumption is lower than 50W and doesn’t peak higher than 200W. Otherwise the savings will be negligible. Another downside of the PicoPSU is the number
of power connectors. You’ll usually have an EPS 4-pin connector,
an ATX connector, one Molex and one SATA connector. And that’s it. However, you can use splitters to connect
more devices, and I’m currently using a Molex Y-splitter
for the SATA backplane On my Supermicro case. It has four WD Pro 7200
RPM hard drives installed in it, And the entire system pulls around 120W while
booting up, Which is well within the 160W spec of my particular
model. For systems with more than four hard drives
or a dedicated GPU, I suggest this 550W Corsair PSU instead. The 2021 model in particular has shown to
be almost as power efficient at low loads as a PicoPSU, And with way more power connectors,
it’s basically the best of both worlds. If you’re building a NAS, you’re obviously
gonna need hard drives. And here you have a choice between 5400 and
7200RPM drives. The 7200RPM drives are usually slightly faster
than the 5400 ones, but also run louder and hotter; and consume
more power. Western Digital, who by the way, sponsored
this video, has both 5400 and 7200 RPM CMR drives in their
Red line up WD Red Plus and WD Red Pro. The WD Red Plus drives consume as little as
3W in idle, and when spinned down, this number goes to
0,4W. The 7200RPM Red Pro drives, apart from being
faster, can also be used in server enclosures with
up to 24 hard drive bays, thanks to their anti-vibration technology. Both WD Red Plus and Pro are CMR drives,
No matter which capacity you go for, and are a great choice for a RAID or ZFS array. Thanks again Western Digital for sponsoring
this video, and now let’s go back to the topic of power
efficiency If you’re trying to build a power efficient,
cool and quiet server, I would definitely recommend going with 5400RPM
drives. You probably won’t notice the speed
difference compared to 7200RPM drives Unless you run a RAID or ZFS array with dozens of drives, But what you will notice is slower spin up times,
Higher temperatures and higher power consumption So unless you absolutely know that you need
7200RPM drives, just go with 5400RPM. However, even 5400RPM drives will still consume
power while spinning It’s not much – a single hard drive will
draw around 3 to 8 watts, but it obviously scales with the number
of hard drives that you have in your system. So no matter how well optimized the rest of
your build is, If you have let’s say 6 or 8 hard drives
spinning all the time, You will see a sharp increase in power consumption
and noise Now my personal solution to this is to
spin down the drives after a certain period of inactivity However, spindown is kind of a controversial topic Some people say that spinning your drives
up and down Causes additional wear and tear,
And can possibly reduce your drive’s lifespan. But some people argue that this is
not an issue anymore with newer hard drives Sminlal on Tom's Hardware forums says: This is not as big a problem as it once was
In the old days when you spun down a drive, The heads would lose their air cushion and come to
rest on the platter as it was slowing down That caused wear Most modern drives use loading ramps
which let the heads be retracted beyond the edge of the disk So that there's no physical contact between them And the platters when the drive is spun down As a result of this the drive specs that I've seen Show ratings on the order of 300,000 stop/start cycles That's a 10-year lifespan if the drives are
stopped and started Every 20 minutes during every hour of every
day, 365 days a year Spinning down a drive brings its power consumption
from 3-8 watts Down to less than half a watt It also completely eliminates the noise, and
provided you don’t spin your drives up and down every 5 minutes,
Shouldn’t have any appreciable impact on your drives’ lifespan. I found that spinning the drives down after 30 minutes of inactivity works pretty well for my case However, spinning down the drives also means
that they’ll have to spin back up Before you can access
the data stored on them. This might not be a big deal with 5400RPM drives, But 7200RPM drives take longer to spin up, And if you’re running something like Unraid
or MergerFS, The drives will spin up sequentially instead
of all at the same time, Which will result in a longer wait before you
can access the files stored on those drives But there is a way you can eat you cake and have it too That is to say, have near-immediate
access to frequently used data and minimize the hard drive power draw and noise. Normally, if you want to have a fast NAS
With instant access times, You have to either create a RAID array
with dozens of drives and keep them all powered on all the time… Or build an SSD-only NAS with no spinning
rust whatsoever. The former requires a beefy server motherboard
and multiple HBA cards, And will also have a massive impact on your
power consumption …And the latter will obviously cost an arm
and a leg, Since SSDs are much more expensive when it comes to the raw dollar per gigabyte value. However, by adding a relatively small SSD
array to your NAS, You can make use of a technique called
"Tiered caching". Tiered caching is the process of combining
fast and slow drives into one array To speed up the file operations You basically tell all applications and services
To write to the SSD array first, And once it gets filled up or you haven’t
accessed the data for a while, the files get moved to the slow drives Unraid, one of the popular NAS operating systems,
actually includes a utility called ‘Mover’ that does exactly that You can set any of your shares to use the cache, and then the Mover will take care of moving the files From the cache array to the slow spinning
disk array regularly If you’re not using Unraid, there’s also
a Python script Written By a Github user elmuz
that basically accomplishes the same thing You can schedule it with cron and customize
some parameters to fit your workflow, such as the target percentage
of drive space and so on. The SSD array you’ll use for caching can
be as small, or as big as you can afford, but in this situation,
bigger is better. If you attempt to write a file that is bigger than
the whole SSD in one go, You’ll probably encounter an input output error. Apart from that, the bigger your array is, The less you’ll have to spin up your hard drives Which means more power savings
and longer drive lifespan. For my NAS, I currently use a RAID0 array of
two 1TB consumer grade NVMe SSDs, And it has been plenty for my use case I am however planning to upgrade it to an
array of four 2TB WD Red SSDs soon though, so make sure to subscribe if you don’t want
to miss that video So that’s basically all the tech tips I’ve
got for you today, I hope you enjoyed this video
