Hi,
Trying to get the power consumption down, so I installed powertop to see the package C-States and CPU C-State usage. I think C-States look fine, as C6 is the lowest my E5-2683 v4 supports, and C6 is heavily used?:
Clocks also look fine, as I switch the governor to shedutil and limited the max clock to 2100 MHz so boosting to the default 3000 MHz is now forbidden:
But when looking at the tunables page there still might be stuff to optimize:
1.) When downtime isn't a problem and I could always connect to IPMI to hard reset the server, it would be OK to disable the NMI watchdog? Would the watchdog help with a graceful shutdown (so good for data integrity) or have I screwed up anyway when the kernel freezes?
2.) What about the SATA Link Power Management? I didn't enable Aggressive Link Power Management in BIOS as far as I know. And my thought was that enabling ALPM isn't that great because this might cause data loss. According to
3.) What does the "Runtime PM ..." mean?
And then I got a Connect-X3 NIC, Intel i350-T4 and a GT710 connected, where the GT710 is passthroughed while the NICs are used by the host.
Is there a way to see if these devices somehow got a power saving mode? And if yes, is there a way to monitor this state?
I for example read threads in this forum that a GPU might waste a lot of power while it isn't passed through because it might always run at full performance while not connected to a guest OS that tells it to clock down. Most of the time my GT710 isn't passed through but the drivers are blacklisted in PVE. So would be interesting to know if it actually runs at full performance while not being used by a VM. As a workaround, I read I could setup a dummy Linux VM with passthrough that I could start when the WinVMs, which usually use it, are shut down. But not sure if that makes sense if I don't if its firmware powers down when not actively used. Not that it's using more power while passthroughed to a idleing linux VM than just being not used at all.
I used hdparm to set the APM and spindown for my HDDs in the past but now I removed the disks which saved a lot of power.
And I set the APM for my SSDs to 1.
And a script is using ipmitool to control the 5 fans, so the fans shouldn't waste power when cooling isn't needed.
And I shutdown guests that I don't need right now.
Are there other things I missed to save some power?
It's still idling at around 80-100W.
Trying to get the power consumption down, so I installed powertop to see the package C-States and CPU C-State usage. I think C-States look fine, as C6 is the lowest my E5-2683 v4 supports, and C6 is heavily used?:
Clocks also look fine, as I switch the governor to shedutil and limited the max clock to 2100 MHz so boosting to the default 3000 MHz is now forbidden:
But when looking at the tunables page there still might be stuff to optimize:
1.) When downtime isn't a problem and I could always connect to IPMI to hard reset the server, it would be OK to disable the NMI watchdog? Would the watchdog help with a graceful shutdown (so good for data integrity) or have I screwed up anyway when the kernel freezes?
2.) What about the SATA Link Power Management? I didn't enable Aggressive Link Power Management in BIOS as far as I know. And my thought was that enabling ALPM isn't that great because this might cause data loss. According to
cat /sys/class/scsi_host/host*/link_power_management_policy all are set to "max_performance". Now I googled a bit and found this article mentioning that there is a new mode "med_power_with_dipm" that should not cause data loss when used with newer SSDs. My SSDs are 8x Intel S3700 and S3710 (ZFS, mdadm and LVM-Thin) so they are not that new. Can someone confirm that setting the link power management to "med_power_with_dipm" should not risk my data with my SSDs? Saving 12W would be nice but not when this might cause data corruption.3.) What does the "Runtime PM ..." mean?
And then I got a Connect-X3 NIC, Intel i350-T4 and a GT710 connected, where the GT710 is passthroughed while the NICs are used by the host.
Is there a way to see if these devices somehow got a power saving mode? And if yes, is there a way to monitor this state?
I for example read threads in this forum that a GPU might waste a lot of power while it isn't passed through because it might always run at full performance while not connected to a guest OS that tells it to clock down. Most of the time my GT710 isn't passed through but the drivers are blacklisted in PVE. So would be interesting to know if it actually runs at full performance while not being used by a VM. As a workaround, I read I could setup a dummy Linux VM with passthrough that I could start when the WinVMs, which usually use it, are shut down. But not sure if that makes sense if I don't if its firmware powers down when not actively used. Not that it's using more power while passthroughed to a idleing linux VM than just being not used at all.
I used hdparm to set the APM and spindown for my HDDs in the past but now I removed the disks which saved a lot of power.
And I set the APM for my SSDs to 1.
And a script is using ipmitool to control the 5 fans, so the fans shouldn't waste power when cooling isn't needed.
And I shutdown guests that I don't need right now.
Are there other things I missed to save some power?
It's still idling at around 80-100W.
Last edited: