Having issues with backups

nawhouse

New Member
Jun 23, 2024
12
0
1
Hello,
I am having a problem with the backup job of a LXC I have set up that is running docker.
It is running on a NFS share on my TrueNAS and is set in HA. I have a 3 node cluster.
When I try to backup the LXC I get an error that there is not enough space.


Code:
ERROR: rsync: [receiver] write failed on "/var/tmp/vzdumptmp1035638_109/data/compose/19/radarr/MediaCover/387/poster.jpg": No space left on device (28)

I am now trying to set the temp backup dir to a SMB share but that is getting new errors.

Code:
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/bin" -> "usr/bin" failed: Input/output error (5)
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/lib" -> "usr/lib" failed: Input/output error (5)
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/lib32" -> "usr/lib32" failed: Input/output error (5)
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/lib64" -> "usr/lib64" failed: Input/output error (5)
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/libx32" -> "usr/libx32" failed: Input/output error (5)
ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/sbin" -> "usr/sbin" failed: Input/output error (5)

I would sure love some help with this guy.

Thanks!
 
Under datacenter does the NFS share show a gray question mark? If not does the information about the share size and contents display properly? If so does it show you have more than enough room to make the transfer?
 
Nope. No gray question, Yes all the correct info is displayed. There is more than 40 TB of free space.
To me it seems like it is trying to write to a temp folder on the SD card running the PVE os before it syncs it over to my PBS server and there is not enough room on it to do so.
 
before I moved the it to the NFS share and when it was on a zfs store and the backup method is snapshot.
 
Hi,
To me it seems like it is trying to write to a temp folder on the SD card running the PVE os before it syncs it over to my PBS server and there is not enough room on it to do so.
yes, if you read the full backup task log it should tell you that it couldn't use snapshot mode and thus is using suspend mode with the (default) temporary directory /var/tmp. You can configure a different tmpdir for the node in /etc/vzdump.conf.
 
Hi,

yes, if you read the full backup task log it should tell you that it couldn't use snapshot mode and thus is using suspend mode with the (default) temporary directory /var/tmp. You can configure a different tmpdir for the node in /etc/vzdump.conf.
Thanks, Flona.
I have tried changing the tmpdir to a mounted NFS or SMB share but I get lots of errors like

ERROR: rsync: [generator] symlink "/mnt/tmpbak/vzdumptmp652633_109/bin" -> "usr/bin" failed: Input/output error (5)

Can you suggest how to overcome that problem?
I have tested making directories and files on the mount from the pve command line and I am able to do so.
 
Is there anything in the system logs/journal relating to those errors? I'd also check the health of the physical disk, e.g. using smartctl. Does it always fail with links or arbitrary files?
 
There does not seem to be anything in the syslogs. It just appears that when it is trying to write the tmp backup to the local HDD that proxmox is installed on it is running out of space. After moving the LXC back to the ZFS local data store the it backups just fine with a snap backup.
I am the VMWare admin for a fairly large company and with all the Broadcom nonsense I am looking at alternitives and it would be a big positive for proxmox if it could support snapshots on NFS and iSCSI storage.

Thanks