Hi everyone,
I wanted to document a power-consumption problem I spent quite a lot of time investigating, because some of the results were very unintuitive and may also be relevant to other people running Proxmox/Linux on older laptops or Haswell systems.
First, an important disclaimer: I am not a Linux/kernel expert. I started this as a home-server project and learned most of this while troubleshooting. So corrections or better technical explanations are very welcome. Also I used a lot of AI, also to summarize this post. I think it is readable quite good.
The interesting part is that I did not just end up throwing powertop --auto-tune at the machine. I tested the relevant devices and settings individually and measured the effect.
The final result was roughly:
At the final 4–5 W idle state I measured approximately 94% Package C6 residency, while Ethernet remained connected and usable.
Relevant hardware:
The successful tests were done on Proxmox VE 9 / Debian Trixie, with a PVE 7.0-series kernel; at the final measurements I was running:
7.0.14-15-pve
The server is intended to run headless with Ethernet and eventually host Home Assistant, OMV, Immich and other services.
The readings initially fluctuated quite a lot. Depending on background activity, display state etc. I saw roughly 11–14 W, but after getting Windows properly idle I could repeatedly see periods around 6–8 W.
That became my reference.
I also logged the system with HWiNFO rather than relying only on the wall meter.
One strange observation actually happened very early.
I thought:
Instead of reducing consumption, at one point the machine went to approximately:
21 W.
CPU usage was still around 1%.
That seemed completely backwards.
I cannot prove the exact internal reason, but in retrospect I suspect that disabling the AMD device in Device Manager prevented the normal hybrid-graphics driver stack from performing the power transitions it normally performs.
In a later HWiNFO log with the AMD driver functioning normally, the dGPU was mostly inactive but occasionally woke briefly. Afterwards the PCIe-related values disappeared again.
That suggested that Windows was not simply leaving the GPU permanently disabled. It appeared to actively manage it, wake it when required and then shut the relevant hardware path down again.
This became important much later.
19.5–20 W, sometimes somewhat higher.
My first assumption was that Linux/Proxmox itself was simply much less efficient on this old laptop.
But CPU usage was tiny.
powertop made this even more confusing.
The individual CPU cores were spending roughly 98–99% of idle time in deep C-states, mostly C7/C7s.
So from my beginner's perspective, the CPU looked perfect:
At this point I did not yet understand the importance of the distinction between core C-states and package C-states.
Linux correctly detected Toshiba/AMD hybrid graphics. The kernel log contained ATPX/PowerXpress related messages and the AMD driver reported runtime power management.
The AMD device itself looked reasonably good:
01:00.0 AMD GPU
runtime_status: suspended
So initially I assumed that the AMD GPU could not be responsible.
Then I checked the PCIe hierarchy.
The topology is essentially:
00:01.0 Intel PEG / PCIe Root Port
|
+--- 01:00.0 AMD Radeon
And this was the important part:
The AMD endpoint could be suspended while the upstream PEG/root port remained active in D0.
So the GPU itself appearing "suspended" did not necessarily mean the whole PCIe branch had reached its lowest-power state.
pcie_port_pm=force
The Linux kernel documentation describes this as forcibly enabling power management of PCIe ports.
After booting with it and allowing runtime PM on the AMD endpoint/root port:
echo auto > /sys/bus/pci/devices/0000:01:00.0/power/control
echo auto > /sys/bus/pci/devices/0000:00:01.0/power/control
the state changed to:
00:01.0 PEG:
runtime_status: suspended
runtime_usage: 0
power_state: D3cold
01:00.0 AMD:
runtime_status: suspended
runtime_usage: 0
power_state: D3cold
Linux documents D3cold as the PCI power state where the device's supply voltage is removed.
The wall measurement immediately changed from roughly:
19.5–20 W
to approximately:
9.2–10 W
This was by far the first major breakthrough.
Nearly ten watts disappeared.
So the original assumption that "Proxmox simply needs ~20 W on this laptop" was clearly wrong.
A very large part of the problem was the PCIe/dGPU power state.
It also made the earlier Windows behaviour much more interesting: Windows apparently had mechanisms capable of putting the hybrid-GPU branch into a much deeper power state automatically.
Now I started investigating CPU Package C-states directly.
This is where I finally understood why the earlier powertop result had been misleading to me.
Linux's intel_idle documentation explains that there are two relevant levels:
core C-states, which concern individual CPU cores, and package C-states, which concern the processor package as a whole and may depend on other components such as graphics, memory controllers or I/O devices.
So it is entirely possible to have:
Core C7: ~99%
while the processor package itself cannot reach a deep Package C-state.
That was basically my situation.
I measured Package residency directly through the Haswell MSRs:
TSC 0x10
Package C2 0x60D
Package C3 0x3F8
Package C6 0x3F9
Package C7 0x3FA
With the normal Ethernet configuration Package C6 was essentially:
0%
Temporarily disable Ethernet.
With Ethernet active I saw roughly:
Package C2: ~32%
Package C3: ~64%
Package C6: 0%
After taking the Ethernet device/link out of the equation:
Package C6: ~49%
That was a strong diagnostic result.
The active Realtek Ethernet path was preventing the processor from reaching Package C6.
Unfortunately, turning Ethernet off is not very useful for a Proxmox server.
So the question became:
The normal kernel driver was:
r8169
Because the alternative r8168-dkms driver is often discussed for these controllers, I tested that too.
This involved installing DKMS/headers and dealing with module signing/Secure Boot.
Eventually r8168 was definitely bound and working.
Result:
No improvement.
Package C6 still remained unavailable with normal active Ethernet.
So I reverted everything and returned to the stock kernel:
r8169
For my hardware, r8168 provided no power-saving advantage.
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
and around the Realtek device:
r8169 ... can't disable ASPM; OS doesn't have ASPM control
This was extremely interesting.
ASPM is PCI Express Active State Power Management — essentially allowing PCIe links themselves to enter low-power states when idle.
But Toshiba's firmware was telling Linux not to control it.
That created an obvious contradiction.
The same physical laptop could reach much lower power levels under Windows.
So either Windows had additional vendor-specific mechanisms, or the firmware restriction was more conservative than what the hardware was actually capable of.
There is even an old Intel discussion involving Haswell laptops and Linux vs Windows Package C-state residency where a very similar class of problem is discussed: Linux respects firmware/BIOS power-management restrictions while Windows on the same hardware can achieve much deeper package residency. It is not evidence for my exact Toshiba, but it was an interesting parallel.
pcie_aspm=force
Important warning:
The kernel documentation explicitly says that pcie_aspm=force enables ASPM even on devices claiming not to support it and warns:
forcing ASPM may cause system lockups.
So I initially used this only as a temporary GRUB test.
After booting with:
pcie_port_pm=force pcie_aspm=force
the runtime ASPM policy could finally be changed:
echo powersave > /sys/module/pcie_aspm/parameters/policy
With Ethernet still connected and working, Package residency changed dramatically:
Package C2 : ~26.8%
Package C3 : ~0.7%
Package C6 : ~69%
Package C7 : 0%
So:
Before:
Ethernet active → Package C6 = 0%
After forced ASPM:
Ethernet active → Package C6 ≈ 69%
This solved the Ethernet/Package-C6 problem without losing network connectivity.
I also tested:
powersupersave
versus:
powersave
There was effectively no advantage in my measurements.
So I kept:
powersave
So there is no L1.1/L1.2 available on this particular Ethernet branch.
It supports the older ASPM states.
After forcing ASPM and selecting powersave, lspci produced a somewhat odd-looking result where the endpoint showed L0s enabled while the root side still appeared as ASPM disabled.
Nevertheless, Package C6 worked reliably in the measurement.
So I would caution against looking only at one LnkCtl line and assuming that tells the entire platform-power story.
The important result for me was the actual Package residency measurement.
I wanted to document a power-consumption problem I spent quite a lot of time investigating, because some of the results were very unintuitive and may also be relevant to other people running Proxmox/Linux on older laptops or Haswell systems.
First, an important disclaimer: I am not a Linux/kernel expert. I started this as a home-server project and learned most of this while troubleshooting. So corrections or better technical explanations are very welcome. Also I used a lot of AI, also to summarize this post. I think it is readable quite good.
The interesting part is that I did not just end up throwing powertop --auto-tune at the machine. I tested the relevant devices and settings individually and measured the effect.
The final result was roughly:
| State | Wall power |
|---|---|
| Windows reference | often ~6-8 W |
| Initial Proxmox idle | ~19-20 W |
| After PCIe/dGPU fix | ~9-10 W |
| Final optimized Proxmox idle | ~4-5 W |
At the final 4–5 W idle state I measured approximately 94% Package C6 residency, while Ethernet remained connected and usable.
Hardware / software
The machine is a Toshiba Satellite L70-B-132.Relevant hardware:
| Component | Hardware / PCI address |
|---|---|
| CPU | Intel Core i7-4710HQ, Haswell |
| iGPU | Intel HD Graphics 4600, 00:02.0, i915 |
| dGPU | AMD Radeon HD 8850M / R9 M265X, 01:00.0 |
| dGPU root port / PEG | 00:01.0 |
| Ethernet | Realtek RTL8111/8168 family, 07:00.0 |
| Ethernet root port | 00:1c.0 |
| WLAN | Intel Wireless 3160, 08:00.0 |
| WLAN root port | 00:1c.1 |
| Card reader | Realtek RTS5229, 09:00.0 |
| Card-reader root port | 00:1c.2 |
| SATA/AHCI | Intel 8 Series/C220, 00:1f.2 |
| SSD | SanDisk SDSSDH3 500 GB |
| USB xHCI | 00:14.0 |
| USB EHCI | 00:1a.0, 00:1d.0 |
| MEI | 00:16.0 |
7.0.14-15-pve
The server is intended to run headless with Ethernet and eventually host Home Assistant, OMV, Immich and other services.
1. Why I started investigating this
Before installing Proxmox I measured the laptop under Windows.The readings initially fluctuated quite a lot. Depending on background activity, display state etc. I saw roughly 11–14 W, but after getting Windows properly idle I could repeatedly see periods around 6–8 W.
That became my reference.
I also logged the system with HWiNFO rather than relying only on the wall meter.
One strange observation actually happened very early.
I thought:
So I disabled WLAN, Bluetooth, optical-drive related hardware and eventually the AMD Radeon.
Instead of reducing consumption, at one point the machine went to approximately:
21 W.
CPU usage was still around 1%.
That seemed completely backwards.
I cannot prove the exact internal reason, but in retrospect I suspect that disabling the AMD device in Device Manager prevented the normal hybrid-graphics driver stack from performing the power transitions it normally performs.
In a later HWiNFO log with the AMD driver functioning normally, the dGPU was mostly inactive but occasionally woke briefly. Afterwards the PCIe-related values disappeared again.
That suggested that Windows was not simply leaving the GPU permanently disabled. It appeared to actively manage it, wake it when required and then shut the relevant hardware path down again.
This became important much later.
2. Initial Proxmox result: almost 20 W
After installing Proxmox the laptop idled at roughly:19.5–20 W, sometimes somewhat higher.
My first assumption was that Linux/Proxmox itself was simply much less efficient on this old laptop.
But CPU usage was tiny.
powertop made this even more confusing.
The individual CPU cores were spending roughly 98–99% of idle time in deep C-states, mostly C7/C7s.
So from my beginner's perspective, the CPU looked perfect:
Yet the wall meter still showed approximately 20 W.
At this point I did not yet understand the importance of the distinction between core C-states and package C-states.
3. The AMD dGPU looked suspended — but something was still wrong
The laptop contains both the Intel HD 4600 and an AMD Radeon.Linux correctly detected Toshiba/AMD hybrid graphics. The kernel log contained ATPX/PowerXpress related messages and the AMD driver reported runtime power management.
The AMD device itself looked reasonably good:
01:00.0 AMD GPU
runtime_status: suspended
So initially I assumed that the AMD GPU could not be responsible.
Then I checked the PCIe hierarchy.
The topology is essentially:
00:01.0 Intel PEG / PCIe Root Port
|
+--- 01:00.0 AMD Radeon
And this was the important part:
The AMD endpoint could be suspended while the upstream PEG/root port remained active in D0.
So the GPU itself appearing "suspended" did not necessarily mean the whole PCIe branch had reached its lowest-power state.
4. First major breakthrough:
I then tested the kernel parameter:pcie_port_pm=force
The Linux kernel documentation describes this as forcibly enabling power management of PCIe ports.
After booting with it and allowing runtime PM on the AMD endpoint/root port:
echo auto > /sys/bus/pci/devices/0000:01:00.0/power/control
echo auto > /sys/bus/pci/devices/0000:00:01.0/power/control
the state changed to:
00:01.0 PEG:
runtime_status: suspended
runtime_usage: 0
power_state: D3cold
01:00.0 AMD:
runtime_status: suspended
runtime_usage: 0
power_state: D3cold
Linux documents D3cold as the PCI power state where the device's supply voltage is removed.
The wall measurement immediately changed from roughly:
19.5–20 W
to approximately:
9.2–10 W
This was by far the first major breakthrough.
Nearly ten watts disappeared.
So the original assumption that "Proxmox simply needs ~20 W on this laptop" was clearly wrong.
A very large part of the problem was the PCIe/dGPU power state.
It also made the earlier Windows behaviour much more interesting: Windows apparently had mechanisms capable of putting the hybrid-GPU branch into a much deeper power state automatically.
5. CPU cores were sleeping, but the CPU package was not
After the dGPU fix the machine was around 9–10 W.Now I started investigating CPU Package C-states directly.
This is where I finally understood why the earlier powertop result had been misleading to me.
Linux's intel_idle documentation explains that there are two relevant levels:
core C-states, which concern individual CPU cores, and package C-states, which concern the processor package as a whole and may depend on other components such as graphics, memory controllers or I/O devices.
So it is entirely possible to have:
Core C7: ~99%
while the processor package itself cannot reach a deep Package C-state.
That was basically my situation.
I measured Package residency directly through the Haswell MSRs:
TSC 0x10
Package C2 0x60D
Package C3 0x3F8
Package C6 0x3F9
Package C7 0x3FA
With the normal Ethernet configuration Package C6 was essentially:
0%
6. Ethernet turned out to be blocking Package C6
The next test was very simple:Temporarily disable Ethernet.
With Ethernet active I saw roughly:
Package C2: ~32%
Package C3: ~64%
Package C6: 0%
After taking the Ethernet device/link out of the equation:
Package C6: ~49%
That was a strong diagnostic result.
The active Realtek Ethernet path was preventing the processor from reaching Package C6.
Unfortunately, turning Ethernet off is not very useful for a Proxmox server.
So the question became:
7. I tested the Realtek
The NIC is an RTL8111/8168-family device.The normal kernel driver was:
r8169
Because the alternative r8168-dkms driver is often discussed for these controllers, I tested that too.
This involved installing DKMS/headers and dealing with module signing/Secure Boot.
Eventually r8168 was definitely bound and working.
Result:
No improvement.
Package C6 still remained unavailable with normal active Ethernet.
So I reverted everything and returned to the stock kernel:
r8169
For my hardware, r8168 provided no power-saving advantage.
8. Then I found the firmware/ASPM contradiction
The kernel log contained this:ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
and around the Realtek device:
r8169 ... can't disable ASPM; OS doesn't have ASPM control
This was extremely interesting.
ASPM is PCI Express Active State Power Management — essentially allowing PCIe links themselves to enter low-power states when idle.
But Toshiba's firmware was telling Linux not to control it.
That created an obvious contradiction.
The same physical laptop could reach much lower power levels under Windows.
So either Windows had additional vendor-specific mechanisms, or the firmware restriction was more conservative than what the hardware was actually capable of.
There is even an old Intel discussion involving Haswell laptops and Linux vs Windows Package C-state residency where a very similar class of problem is discussed: Linux respects firmware/BIOS power-management restrictions while Windows on the same hardware can achieve much deeper package residency. It is not evidence for my exact Toshiba, but it was an interesting parallel.
9. Second major breakthrough:
I therefore tested:pcie_aspm=force
Important warning:
The kernel documentation explicitly says that pcie_aspm=force enables ASPM even on devices claiming not to support it and warns:
forcing ASPM may cause system lockups.
So I initially used this only as a temporary GRUB test.
After booting with:
pcie_port_pm=force pcie_aspm=force
the runtime ASPM policy could finally be changed:
echo powersave > /sys/module/pcie_aspm/parameters/policy
With Ethernet still connected and working, Package residency changed dramatically:
Package C2 : ~26.8%
Package C3 : ~0.7%
Package C6 : ~69%
Package C7 : 0%
So:
Before:
Ethernet active → Package C6 = 0%
After forced ASPM:
Ethernet active → Package C6 ≈ 69%
This solved the Ethernet/Package-C6 problem without losing network connectivity.
I also tested:
powersupersave
versus:
powersave
There was effectively no advantage in my measurements.
So I kept:
powersave
10. Interesting ASPM detail on this Realtek link
The Realtek endpoint/root-port pair does not advertise PCIe L1 PM Substates.So there is no L1.1/L1.2 available on this particular Ethernet branch.
It supports the older ASPM states.
After forcing ASPM and selecting powersave, lspci produced a somewhat odd-looking result where the endpoint showed L0s enabled while the root side still appeared as ASPM disabled.
Nevertheless, Package C6 worked reliably in the measurement.
So I would caution against looking only at one LnkCtl line and assuming that tells the entire platform-power story.
The important result for me was the actual Package residency measurement.