Hi,
Since I upgraded the packages on my Proxmox host backups have been failing. During the upgrade process apt asked whether to keep a file that was manually edited or replace it with the package version. I chose to keep the manually edited one, which might have been a mistake. I believe it was the vzdump.conf file, in which I defined a temp dir on the local storage (backups are being sent to a NFS share).
Ever since then backups have been failing with errors along the lines of:
Next to that the read/write speeds of the backups slow down considerably over time, they start of quite fast but then slow down to near 0.
This is the output of pveversion -v:
Since I upgraded the packages on my Proxmox host backups have been failing. During the upgrade process apt asked whether to keep a file that was manually edited or replace it with the package version. I chose to keep the manually edited one, which might have been a mistake. I believe it was the vzdump.conf file, in which I defined a temp dir on the local storage (backups are being sent to a NFS share).
Ever since then backups have been failing with errors along the lines of:
Warning: unable to close filehandle GEN353 properly: Input/output error at /usr/share/perl5/PVE/VZDump/QemuServer.pm line 770.
zstd: /*stdout*\: Input/output error
ERROR: Backup of VM 101 failed - zstd --rsyncable --threads=1 failed - wrong exit status 1Next to that the read/write speeds of the backups slow down considerably over time, they start of quite fast but then slow down to near 0.
INFO: 0% (735.9 MiB of 80.0 GiB) in 3s, read: 245.3 MiB/s, write: 187.8 MiB/s
INFO: 2% (1.6 GiB of 80.0 GiB) in 6s, read: 311.8 MiB/s, write: 204.1 MiB/s
INFO: 3% (2.4 GiB of 80.0 GiB) in 10s, read: 198.5 MiB/s, write: 193.1 MiB/s
INFO: 4% (3.3 GiB of 80.0 GiB) in 14s, read: 221.1 MiB/s, write: 212.6 MiB/s
INFO: 5% (4.1 GiB of 80.0 GiB) in 19s, read: 173.3 MiB/s, write: 165.2 MiB/s
INFO: 6% (4.9 GiB of 80.0 GiB) in 25s, read: 133.5 MiB/s, write: 129.1 MiB/s
INFO: 7% (5.6 GiB of 80.0 GiB) in 30s, read: 152.2 MiB/s, write: 146.4 MiB/s
INFO: 8% (6.6 GiB of 80.0 GiB) in 35s, read: 187.0 MiB/s, write: 186.8 MiB/s
INFO: 9% (7.4 GiB of 80.0 GiB) in 39s, read: 203.9 MiB/s, write: 196.9 MiB/s
INFO: 10% (8.1 GiB of 80.0 GiB) in 43s, read: 192.1 MiB/s, write: 190.4 MiB/s
INFO: 11% (8.9 GiB of 80.0 GiB) in 47s, read: 205.7 MiB/s, write: 203.7 MiB/s
INFO: 12% (9.6 GiB of 80.0 GiB) in 51s, read: 184.9 MiB/s, write: 183.7 MiB/s
INFO: 13% (10.4 GiB of 80.0 GiB) in 10m 39s, read: 1.3 MiB/s, write: 1.2 MiB/s
INFO: 14% (11.2 GiB of 80.0 GiB) in 16m 6s, read: 2.5 MiB/s, write: 2.5 MiB/s
INFO: 15% (12.0 GiB of 80.0 GiB) in 18m 40s, read: 5.3 MiB/s, write: 5.3 MiB/s
INFO: 16% (12.8 GiB of 80.0 GiB) in 23m 50s, read: 2.7 MiB/s, write: 2.6 MiB/s
INFO: 17% (13.6 GiB of 80.0 GiB) in 28m 1s, read: 3.2 MiB/s, write: 3.2 MiB/s
INFO: 18% (14.4 GiB of 80.0 GiB) in 33m 43s, read: 2.4 MiB/s, write: 2.4 MiB/s
INFO: 19% (15.2 GiB of 80.0 GiB) in 39m 43s, read: 2.3 MiB/s, write: 2.2 MiB/s
INFO: 20% (16.0 GiB of 80.0 GiB) in 45m 44s, read: 2.3 MiB/s, write: 2.3 MiB/s
INFO: 21% (16.8 GiB of 80.0 GiB) in 51m 4s, read: 2.6 MiB/s, write: 2.5 MiB/sThis is the output of pveversion -v:
proxmox-ve: 7.2-1 (running kernel: 5.15.39-1-pve)
pve-manager: 7.2-7 (running version: 7.2-7/d0dd0e85)
pve-kernel-5.15: 7.2-6
pve-kernel-helper: 7.2-6
pve-kernel-5.13: 7.1-9
pve-kernel-5.4: 6.4-13
pve-kernel-5.15.39-1-pve: 5.15.39-1
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.4.166-1-pve: 5.4.166-1
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.4.143-1-pve: 5.4.143-1
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-3
libpve-storage-perl: 7.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.3-1
proxmox-backup-file-restore: 2.2.3-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-2
pve-ha-manager: 3.3-4
pve-i18n: 2.7-2
pve-qemu-kvm: 6.2.0-11
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1