pvesh get /nodes/{node}/qemu & qm list very slow since Proxmox 9

[Decco1337]

I have upgraded several hosts from Proxmox 8 to 9 and since then I have very long loading times when I use the command

pvesh get /nodes/{node}/qemu

and the same for

qm list

The problem is it takes 40-60s to get a result on different hosts, also with hosts with >50 VMs and >60% CPU usage. At Proxmox 8 it just took few (up to 3s) seconds.

How to solve this? Is this an known issue?

 

[fabian]

are you using KSM? if so, this might be https://bugzilla.proxmox.com/show_bug.cgi?id=6935

 

[Decco1337]

are you using KSM? if so, this might be https://bugzilla.proxmox.com/show_bug.cgi?id=6935

Yes I do

 

[Decco1337]

are you using KSM? if so, this might be https://bugzilla.proxmox.com/show_bug.cgi?id=6935

Any ETA when this will be fixed? What do you recommend? Switching to another kernel or downgrading to Proxmox 8?

 

[fabian]

the change is in qemu-server ,not the kernel, so the only workaround atm is to disable KSM (either globally or selectively for certain VMs).

 

[Decco1337]

the change is in qemu-server ,not the kernel, so the only workaround atm is to disable KSM (either globally or selectively for certain VMs).

Unfortunately disabling KSM is not a solution for me. So downgrading to Proxmox 8 is the only way?

 

[fabian]

you cannot downgrade to PVE 8.x (other than by reinstalling).

you could try the RFC patch I just sent to pve-devel:

https://lore.proxmox.com/pve-devel/20251125135107.561633-1-f.gruenbichler@proxmox.com/T/#u

depending on your current KSM usage pattern, you might want to adapt the threshold value to something lower. feedback would be appreciated!

 

[Decco1337]

you cannot downgrade to PVE 8.x (other than by reinstalling).

you could try the RFC patch I just sent to pve-devel:

https://lore.proxmox.com/pve-devel/20251125135107.561633-1-f.gruenbichler@proxmox.com/T/#u

depending on your current KSM usage pattern, you might want to adapt the threshold value to something lower. feedback would be appreciated!

Yeah meant by reinstalling the system, is not a big deal to go back to Proxmox 8 for me

 

[servereasy]

I tried to disable KSM with:
systemctl stop ksmtuned
echo 2 > /sys/kernel/mm/ksm/run


but the problem persists:
time qm list
<...>
real 0m38.411s
user 0m1.136s
sys 0m37.242s

 

[fabian]

I tried to disable KSM with:
systemctl stop ksmtuned
echo 2 > /sys/kernel/mm/ksm/run


but the problem persists:
time qm list
<...>
real 0m38.411s
user 0m1.136s
sys 0m37.242s

yes, even disabling KSM by unmerging does not get rid of all traces of KSM inside the memory mappings, and unfortuantely, this particular slowness requires the VM to be stopped and started again afterwards to get rid of all memory mappings that have been merged in the past.

 

[Decco1337]

you cannot downgrade to PVE 8.x (other than by reinstalling).

you could try the RFC patch I just sent to pve-devel:

https://lore.proxmox.com/pve-devel/20251125135107.561633-1-f.gruenbichler@proxmox.com/T/#u

depending on your current KSM usage pattern, you might want to adapt the threshold value to something lower. feedback would be appreciated!

Tried the patch, but the problem still exists. It is slightly faster but still too slow.

 

[Decco1337]

The loading times halved with it, but it still takes around 30s for qm list except the system has a high or low load.