Low power server

[tonycav]

Hi, I would like to buy a low-power home server but unfortunately the market offers little at the moment
My needs are these
VM: Home Assistant - OMV - Debian xfce
LXC: MQTT - Jellyfin
1 NVME + 1 SSD + 1 3.5 HD + 32 GB RAM
I found a Dell Optiplex i5-9500 (not i5-9500T) and wanted advice on any consumption scenarios in relation to this configuration

Thank you for your time

 

[wallacio]

With that configuration, you can expect to pull about 30-40W at idle.

 

[tonycav]

does the processor have too high a tdp?

 

[Dunuin]

Its usually not about the processor. TDP is only for dimensioning the cooling and is sometimes used as a guide on how much power the CPU can consume under max load. It doesn't tell you anything how much power it is used under idle or low load. What you probably care more about in a homelab is the idle consumption which isn't adversised anywhere. I can run a proper 4U server at 20W idle while a NUC-like might consume 100W under load.
What helps is to have as few features as possible. Low end chipset, as less mainboard features as possible, efficient low-voltage CPU doesn't hurt, no old hardware, as less disks as possible, no HDDs, a good efficient PSU that is not over or under dimensioned, no additional PCIe cards or USB stuff, no RGB.

 

[scyto]

My 3 x 13th Gen NUCs (promox cluster with 5 VMs ) seem to be pulling between 18 to 29 (at least thats what my PDU is reporting)
My 3 x 10th Gen NUCs pull 10 to 11 watts (think hyper-v server and one or two VMs)

All NUCs are i7's, have one SSD and one NVME

Not suggesting you get NUCs, just giving you a baseline.

 

[UdoB]

Just another random data point:

• MinisForum HM80 + two SSD + M2.NVMe + 64 GiB Ram + both NICs connected + PVE idle (!) = 9 Watt

I've pushed it to consume up to 50 Watt with stress-ng and my personal usecase consumes ~25 Watt. Ymmv...

 

[xnd]

I have very good candidate for low idle power consumption minimalistic, powerful home-server:

It has idle power consuption only 2—3 W :-O

HW:
MB: Asus Pro H610T D4-CSM
CPU: Intel i3-13100 (TDP 60W)
RAM: 1x Crucial 32GB DDR4-3200 SODIMM (CT32G4SFD832A) //max. is 2x32GB
SSD: 1x m.2 Samsung 980 Pro 2TB
PSU: 12V 150W from AKASA https://www.akasa.com.tw/update.php?tpl-product/product.detail.tpl.tpl&model-AK150-02K

OS: Debian 12 with DietPi, kernel 6.1.0-13-amd64 (upgrded to 6.5.0-10-amd64 and is ok too), keyboard and LCD unplugged (only LAN plugged with 1 active SSH session).

Power consumption is measured on DC side (12V, yes this board can be powered with 12V).

It’s fanless minimalistic build.

idle: 2—3 W
load: 50—60W (sit can be adjusted in BIOS to TDP 20W for example)


Here is measurement of power consumption on DC side:

- (only LAN plugged with 1 active SSH session. I don't have Proxmox yet).

All power saving features are enabled in BIOS (C-States, ASPM). In OS powertop was applied and script for enabling ASMP on PCIE devices.

I know, the mainboard is not extendable, no PCI slots, but for me is good enough.
I was able to buy MB+CPU under 180 Eur (which is price of new Odroid H3).

Spoiler: more details...

neofetch:

dietpi@pc:~$ neofetch
       _,met$$$$$gg.          dietpi@pc
    ,g$$$$$$$$$$$$$$$P.       ---------
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux trixie/sid x86_64
 ,$$P'              `$$$.     Kernel: 6.5.0-4-amd64
',$$P       ,ggs.     `$$b:   Uptime: 1 hour, 25 mins
`d$$'     ,$P"'   .    $$$    Packages: 249 (dpkg)
 $$P      d$'     ,    $$P    Shell: bash 5.2.15
 $$:      $$.   -    ,d$$'    Terminal: /dev/pts/0
 $$;      Y$b._   _,d$P'      CPU: 13th Gen Intel i3-13100 (8) @ 3.400GHz
 Y$$.    `.`"Y$$$$P"'         GPU: Intel Alder Lake-S GT1 [UHD Graphics 730]
 `$$b      "-.__              Memory: 191MiB / 31459MiB
  `Y$$
   `Y$$.                                   
     `$$b.                                 
       `Y$$b.
          `"Y$b._
              `"""

powertop outputs (and other details):

and another power measurements:


! Note that photo and screenshot of Shelly were not taken in the same time! Power consumption was not constant number but jumping values in time.

 

[fabitaly67]

I Want to share my recent experience with a new Proxmox Home server just assembled and actually in succesful operation since two weeks.

The Hardware server is composed by:

- Rack 2U Frontal connections Compact depth (only 380mm) -Low Profile PCIe Slots 200W Power Supply
- Mother board ASROCK Z790M PG LIGHTNING/D4 socket 1700
- CPU I9-14900T Intel 24cores 36T - TDP 35 Watt with Noctua Compact silent fan/cooler
- 128GB RAM DDR4
- NVME SSD PCIe 6,4 TB - Intel P4618
- 2 HDD 12TB Seagate Ironwolf
- extra NIC 10Gbe RJ45 dual port

Total cost with new and used Parts, around 2200Euros

Total consumption vary beetween 70 and 85 Watts

This Proxmox machine hosts 13 VMs (Windows7/10/11, Linux, Firewalls, IP PBX) with total of 65 Vcpu/Vcore allocated and CPU commitment variable from 5 to 40% average

I Think, this could be the most efficent and powerful Proxmox Host you could desire.

At your disposal to share opinions.
Best regards

FAB

 

[celemine1gig]

A few points, concerning low power:
- Consider that 3.5" drives will usually consume way more power, even in idle, when compared to 2.5" hdds and especially any kind of flash based storage (be it NVMes or SATA or SAS SSDs).
- So, if you do not need the highest storage capacity at all times, then having a running setup with less storage and frequent backups to (only then powered) bigger storage with 3.5" drives, might make sense.

- Hardware and BIOS-Settings and SW need to match to reach the best results.
- If only one is suboptimal, so will be your power numbers.

- Baremetal tends to consume less, than a virtualized setup.
- Virtualization has an overhead of its own. Add that to the fact, that on your virtualization host, most likely not every piece of hardware will be handled by a specific driver for it, at all times. ==> higher power numbers!

Depending on where you are located on the globe (concerning availability), there are nice systems from Dell, Lenovo and especially Fujitsu, that CAN be very power efficient.
Talking about Server hardware, the Fujitsu TX1320 M3/M4/M5 are a good example of how real server appliactions and features can go hand in hand with (relatively to all the features) low power.

I would suggest, that you think about things like:
- Should I have ECC memory (ZFS?) ?
- How much compute performance and how much storage capacity do I absolutely need 24/7?
- Maybe an Atom or APU would be enough for your use case?
- ...

 

[Dunuin]

any kind of flash based storage (be it NVMes or SATA or SAS SSDs).

Depends...not always true for Enterprise SSDs. Those can actually consume more power than a 3.5" 5400 RPM HDD and not always got power saving features. Not unusual that they consume 5W at idle and 20W when active.

- Baremetal tends to consume less, than a virtualized setup.

Also depends. I've seen Windows and FreeBSD consuming less power when virtualized on a well optimized PVE than running those bare metal on the same hardware. Windows for example is terrible at handling my Ryzen 5800X (without any spinning rust and when idling still 90W in power save mode and 110W in balanced mode).

Should I have ECC memory (ZFS?) ?

Yes, this is usually the main problem. Once it is power efficient, it usually won't support ECC RAM.

 

[celemine1gig]

Depends...not always true for Enterprise SSDs. Those can actually consume more power than a 3.5" 5400 RPM HDD and not always got power saving features. Not unusual that they consume 5W at idle and 20W when active.

That is absolutely correct. However I doubt, that Enterprise SSDs is what tonycav had in mind.
But good point as well:
What kind of storage class (Consumer, Mid-Range-Workstation class or even Enterprise) do I intend to use for my data. Most of the time the power-consumption goes up with each step in reliability, like Dunuin hinted at.

Also depends. I've seen Windows and FreeBSD consuming less power when virtualized on a well optimized PVE than running those bare metal on the same hardware. Windows for example is terrible at handling my Ryzen 5800X (without any spinning rust and when idling still 90W in power save mode and 110W in balanced mode).

That showcases excatly what I meant by:

- Hardware and BIOS-Settings and SW need to match to reach the best results.
- If only one is suboptimal, so will be your power numbers.

If your hardware isn't power-efficient your screwed. If your hardware is power-efficient, but your BIOS settings are crap, you are most likely also screwed (at least up to a point). If your HW, and BIOS settings are top notch, concerning power-efficiency, but your OS and drivers don't handle power-management well, you guessed it: Srewed again!
Building and running power-efficient setups is not as easy as it might sound.
A system might run super efficient for month, but a single SW-update might really rain on your parade, making it way less efficient.

Yes, this is usually the main problem. Once it is power efficient, it usually won't support ECC RAM.

I cannot really comment on the AMD side of things, as I only rarely used recent AMD HW. However, on the Intel side, the Core-Architecture Xeons (Skylake, KabyLake, CoffeeLake, ...) were always a good middle ground, where you could get features like ECC-support and good power-efficiency.
Atom also supports ECC nowadays (look at Elkhartlake X6000 series) with certain SKUs. The big Xeons (E5 and Scalable) are out of the question for home use and power efficiency.

 

[louie1961]

I may be late to this thread, but if you haven't made a purchase yet, you could always consider a 1L PC. I have an HP Elitedesk 800 G9 as my main Proxmox node. It has two m.2 slots, so I have Proxmox installed in a ZFS mirror on 2 NVME drives. HP makes an optional 2.5gbe and a 10gbe NIC that go into the FlexIO port on the back. I run mine with the 10gbe NIC, 64 gb of memory and a i5-12500T processor It consumes around 12 watts at idle I also have a N100 NUC type device as a secondary Proxmox node. It only has 16gb of RAM and I can spin up 7 or 8 VMs easily. That device idles at 6 watts.

 

[celemine1gig]

Small addition, now that I think of it: Even the E3900 ApolloLake SOCs can do ECC. If those are up to the task, in general, is a different question.

And concerning the original question from tonycav:
I could guess, that your system might idle with something like 12W - 18W, but there are too many factors at play here to be precise. The choice in storage devices alone might make a considerable difference.

 

[Dunuin]

I cannot really comment on the AMD side of things, as I only rarely used recent AMD HW. However, on the Intel side, the Core-Architecture Xeons (Skylake, KabyLake, CoffeeLake, ...) were always a good middle ground, where you could get features like ECC-support and good power-efficiency.
Atom also supports ECC nowadays (look at Elkhartlake X6000 series) with certain SKUs. The big Xeons (E5 and Scalable) are out of the question for home use and power efficiency.

Jep, but usually people buy those MiniPCs or MiniITX boards based on mobile chipsets if they want something highly integrated and power efficient. And those all lack the ability to support ECC. Except for those Atoms for industrial applications and those deliver terrible bang for the buck.
What I really would like to see is something with 8 or 16 E-Cores, ECC, 2x M.2, 2+x SATA, dual channel for 64GB RAM and 2+ NICs...

 

[celemine1gig]

What you are decscribing sounds a bit like this: https://www.cnx-software.com/2024/0...c-features-four-10-gbe-and-five-2-5gbe-ports/

 

[Dunuin]

What you are decscribing sounds a bit like this: https://www.cnx-software.com/2024/0...c-features-four-10-gbe-and-five-2-5gbe-ports/

Yes, but thats a refresh of a 7 year old Atom CPU that is slow as hell and very expensive. Not comparable to those cheap N100 boxes where the E-Cores still beat my fastest E5 v4 Xeons in singlethreaded performance while still only needing 1/14 of the electricity.

 

[emunt6]

I'm using this hardware (Proxmox installed on it, VPN Server in VM):
-HPE Thinclient T620 Plus

https://www.servethehome.com/hp-t620-plus-thin-client-and-firewall-vpn-appliance/

It has better CPU than the "HPE MicroServer GEN8".

I did not find any better/cheaper hardware with low power.

 

[frijsdijk]

I've recently installed proxmox on a Odroid H3+ with 32GB of ram. I've got a sata extender board in the m.2 slot so I have 10 sata connections. Currently 5 1TB SSD's connected, everything on ZFS. Currently has 5 VM's (Linux) running of which two have 12GB of memory and one those VMs has 2 cores, the rest is all 4GB and a pfsense/router/firewall/vpn with 1GB ram. They are all actually doing something and so far it's been running really good. Power draw is about 10-15 Watts.

 

[zenowl77]

you could try a lenovo tiny pc if you want ultra low power and a small unit,

the AMD ryzen models for example, you can get a ryzen 3 5350GE with DDR4 (usually comes with 8gb but i think they usually support 32-64gb ram),

this cpu will provide about 38% more performance than the i5 you listed while using 45% less power (35w instead of 65w the intel uses) and because of idle states etc it should overall consume far less, these tiny pcs are practically the size of your hand and support usually either 2x nvme or 1xnvme + 1xSata, you would of course need to plug the 3.5in hdd via an external bay over usb 3.0 at that point but some models also support a single PCIE 1 slot card, which has the potential to be used for a gpu (as some include gpus such as a p600-p1000, etc) but can be also used for other add in cards like usb 3.2 / USB-C, dual port NIC, etc which should give optimal performance for any sata drive or expansion options for vms.

if you get a 2-4+ bay external enclosure it is the perfect size to sit on top and plug right in, which also gives ample room for expansion

the cpus igpu should also work for transcodes relating to jellyfin

another alternative of course is also any SSF desktop, i would simply look up the cpus of the options you find to see passmark scores and go with the best score for watts you can find or check out the wiki list of cpu models and find the best wattages and look for models including them, usually amd ryzens have really great performance/watt ratios vs intels which seem to focus on one or the other.

 

[alexskysilk]

I did not find any better/cheaper hardware with low power.

these are 6 years old, and still command 150-200 on ebay. If you dont need 5 interfaces, there are plenty of 1L options with a core i5-8500t (6 core) for less then $100; if you do want something with nics for router purposes, the topton and clones with an n5100/5105 are more powerful for less money, and are more power efficient.