pvetsatd uptime time and cpu time

[guerby]

Hi,

I have a few nodes of my PVE 9 (up to date) cluster with very high cumulative cpu time on the pvestatd process (one nearly 1 cpu continual use)

In the journalctl -u pvestatd I get "pvestatd[4044694]: status update time (20.938 seconds)"

Previously in the forum I found:
* https://forum.proxmox.com/threads/p...log-status-update-time-10-733-seconds.107692/
* https://forum.proxmox.com/threads/pvestatd-15200-status-update-time.31227/

I ran "time pvesm status" and it's about 1 second (we have PBS and rbd through network)

Any idea of what to look at?

My guess is that scanning the 50 running VMs on this node has some issue (KVM monitor slow, may be guest agent not responding if pvestatd is trying to talk to it?)

But may be pvestatd does other things I'm not aware of.

Thanks!

 

[PwrBank]

What kind of backing storage are the VMs using?

 

[guerby]

What kind of backing storage are the VMs using?

All VMs are using proxmox created/managed ceph rbd storage on this cluster (56 OSD on 8 nodes, all enterprise SSD, 3000TB raw, 60% used), no issue for several years now

 

[fiona]

Hi,
if you use KSM, it might be: https://bugzilla.proxmox.com/show_bug.cgi?id=6935 which is fixed in qemu-server >= 9.1.4

 

[guerby]

Hi,
if you use KSM, it might be: https://bugzilla.proxmox.com/show_bug.cgi?id=6935 which is fixed in qemu-server >= 9.1.4

Thanks for your quick answer!

Yes ksm is active on this server shaving 60.22 GB of 399.32 used / 503 GiB total

Since we have community subscription I guess we'll have to wait a bit longuer for qemu 9.1.4, I'll report here and in the bugzilla once it's installed

BTW is there a way on a single node to update the underlying qemu of a VM without restarting the VM ? May be like a fake migration to localhost?

 

[fiona]

BTW is there a way on a single node to update the underlying qemu of a VM without restarting the VM ? May be like a fake migration to localhost?

Not usable/exposed in Proxmox VE yet, but upstream has integrated support for this recently (with QEMU 10.0).
Relevant feature request: https://bugzilla.proxmox.com/show_bug.cgi?id=3303
Upstream docs: https://gitlab.com/qemu-project/qemu/-/raw/master/docs/devel/migration/CPR.rst

 

[fiona]

But you don't need to restart/migrate the VM for qemu-server updates. Just for pve-qemu-kvm updates to pick up the new QEMU binary version.