Hello all,
I've read in various places that a PCI device can draw more power when "just sitting" in PVE than when passed through to a VM, the reason for this being that the drivers installed inside of the VM (which control the device) can bring it to a more efficient power state. However, I am noticing exactly the opposite, and I cannot figure out how to fix it.
More concretely: I have a PVE (v9.0.5) host with an Intel iGPU (Alder Lake Iris Xe) and an Nvidia dGPU (Geforce RTX 3050 Ti Mobile). I have two VMs running Ubuntu 24.04 LTS, each receiving either the iGPU or the dGPU. Naturally, I installed the necessary drivers, and things like hardware transcoding work just fine. However, when I look at powertop on the PVE host, I see that both devices are 100% in use, and that the package power does not do better than C3. On the VM with the Intel iGPU, I do see that the device resides in RC6 100% of the time (0% "powered on", 0% RC6p/RC6pp). The VM with the Nvidia dGPU only shows 100% device usage with no useful information on the power state it resides in. nvidia-smi claims that the power draw of the device is 3W.
What does this mean for the overall power draw? When both VMs are online, the power draw is around 50W. When I turn off the Intel iGPU VM, this drops to ~35W. Turning off the Nvidia dGPU VM brings this down further to ~25W. When both VMs are offline, powertop in the host reports 0% device usage and package power states in C6 and C8. Powertop auto-tune doesn't seem to do anything.
I would like to understand why this happens, and why the VM guests cannot reduce the power draw with dedicated drivers installed? Is there a way to save my setup?
I've read in various places that a PCI device can draw more power when "just sitting" in PVE than when passed through to a VM, the reason for this being that the drivers installed inside of the VM (which control the device) can bring it to a more efficient power state. However, I am noticing exactly the opposite, and I cannot figure out how to fix it.
More concretely: I have a PVE (v9.0.5) host with an Intel iGPU (Alder Lake Iris Xe) and an Nvidia dGPU (Geforce RTX 3050 Ti Mobile). I have two VMs running Ubuntu 24.04 LTS, each receiving either the iGPU or the dGPU. Naturally, I installed the necessary drivers, and things like hardware transcoding work just fine. However, when I look at powertop on the PVE host, I see that both devices are 100% in use, and that the package power does not do better than C3. On the VM with the Intel iGPU, I do see that the device resides in RC6 100% of the time (0% "powered on", 0% RC6p/RC6pp). The VM with the Nvidia dGPU only shows 100% device usage with no useful information on the power state it resides in. nvidia-smi claims that the power draw of the device is 3W.
What does this mean for the overall power draw? When both VMs are online, the power draw is around 50W. When I turn off the Intel iGPU VM, this drops to ~35W. Turning off the Nvidia dGPU VM brings this down further to ~25W. When both VMs are offline, powertop in the host reports 0% device usage and package power states in C6 and C8. Powertop auto-tune doesn't seem to do anything.
I would like to understand why this happens, and why the VM guests cannot reduce the power draw with dedicated drivers installed? Is there a way to save my setup?