Backups to NFS share failing

[simplify]

Hi All,

Recently I have had my backups failing to my NFS shares, and I am not sure if the issues have been introduced as part of recent updates to my Proxmox nodes or my Synology DSM version.

Has anyone else seen this issue before and or may know how I can resolve?

NB: I have built new Proxmox Nodes, I have a new cluster of two nodes (PVE 8.2). I have two Synology NAS with the latest DSM installed.

Here is the log from the latest test backup

{{guestname}}
INFO: starting new backup job: vzdump 104 --notes-template '{{guestname}}' --notification-mode auto --node pve02 --storage NFS-ISOs-Backups --mode snapshot --remove 0 --compress zstd
INFO: Starting Backup of VM 104 (lxc)
INFO: Backup started at 2024-05-02 09:52:23
INFO: status = running
INFO: CT Name: pihole-2
INFO: including mount point rootfs ('/') in backup
INFO: mode failure - some volumes do not support snapshots
INFO: trying 'suspend' mode instead
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: CT Name: pihole-2
INFO: including mount point rootfs ('/') in backup
INFO: temporary directory is on NFS, disabling xattr and acl support, consider configuring a local tmpdir via /etc/vzdump.conf
INFO: starting first sync /proc/4766/root/ to /mnt/pve/NFS-ISOs-Backups/dump/vzdump-lxc-104-2024_05_02-09_52_23.tmp
ERROR: Backup of VM 104 failed - command 'rsync --stats -h --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/4766/root//./ /mnt/pve/NFS-ISOs-Backups/dump/vzdump-lxc-104-2024_05_02-09_52_23.tmp' failed: exit code 23
INFO: Failed at 2024-05-02 09:56:20
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors