VM CPU usage during backup

[squirell]

Hi!

Why VM CPU usage(tab "Summary") increase during backup create (to PBS)? It's normal?

backup mode: snapshot
Guest agent not used

guest: windows 2016 server

pveversion:

proxmox-ve: 6.4-1 (running kernel: 5.4.106-1-pve)
pve-manager: 6.4-6 (running version: 6.4-6/be2fa32c)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-2
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.6-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-5
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-3
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1

 

[dietmar]

Backup needs to compute data checksums and compression, and uses TLS (encryption) to send data to the backup server. Those things needs some CPU power .. .

 

[Angelo]

Hi,

We are busy having the same sorts of issues - to the point that a bunch of VM CPU loads hit 100% utilisation during backups (or close to that..)

As regards the diagnostics on this, I get what @dietmar is saying about computing data checksums and compression, but surely this wouldn't impact the VM CPU but the HOST CPU?

Also, I have a fairly strong suspicion that this may somehow be worse (?) when the PBS datastore is a rather busy NFS environment - we're going to test out that hypothesis shortly as follows:

We've elected to install PBS onto a bhyve instance inside TrueNAS with a 200TB RAIDZ2 pool and set up a ZFS volume on TrueNAS which is mounted as block storage by the bhyve PBS instance. That (hopefully) give us the best of all worlds and should take away the issue of NFS - if that works then I think we can narrow this down as being (partly) due to NFS datastores..

We'll provide feedback as soon as we've done some testing..

Kind regards,

Angelo.