[SOLVED] Container began failing backup - low disk space

[bigun89]

Here's the output of the command via web console:

INFO: starting new backup job: vzdump 505 --node homeproxmox --mode snapshot --notes-template '{{guestname}}' --storage PBS --remove 0
INFO: Starting Backup of VM 505 (lxc)
INFO: Backup started at 2022-11-21 08:25:47
INFO: status = running
INFO: CT Name: KF2-ct
INFO: including mount point rootfs ('/') in backup
INFO: mode failure - some volumes do not support snapshots
INFO: trying 'suspend' mode instead
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: CT Name: KF2-ct
INFO: including mount point rootfs ('/') in backup
INFO: starting first sync /proc/1375845/root/ to /var/tmp/vzdumptmp1377168_505
ERROR: Backup of VM 505 failed - command 'rsync --stats -h -X -A --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/1375845/root//./ /var/tmp/vzdumptmp1377168_505' failed: exit code 11
INFO: Failed at 2022-11-21 08:29:34
INFO: Backup job finished with errors
TASK ERROR: job errors

Ran the command manually:

root@homeproxmox:~# rsync --stats -h -X -A --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/1375845/root//./ /var/tmp/vzdumptmp1377168_505
rsync: [receiver] write failed on "/var/tmp/vzdumptmp1377168_505/home/kf2/kf2/KFGame/BrewedPC/Packages/Characters/Cosmetics/CHR_CosmeticSetPSN02_MESH.upk": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(378) [receiver=3.2.3]
rsync error: errors with program diagnostics (code 13) at log.c(245) [sender=3.2.3]

df -h:

root@homeproxmox:/var/tmp# df -h
Filesystem            Size  Used Avail Use% Mounted on
udev                   32G     0   32G   0% /dev
tmpfs                 6.3G   42M  6.3G   1% /run
/dev/mapper/pve-root   27G   27G     0 100% /
tmpfs                  32G   43M   32G   1% /dev/shm
tmpfs                 5.0M     0  5.0M   0% /run/lock
/dev/sda2             511M  340K  511M   1% /boot/efi
/dev/sdb1             547G  364G  155G  71% /mnt/intflash
/dev/fuse             128M   24K  128M   1% /etc/pve
tmpfs                 6.3G     0  6.3G   0% /run/user/0

Yup, root filesystem is low on space.... but why is it only 27GB? I have a 120GB OS array dedicated to it.

I have some extra space on another array I use for storing my VM's and containers. Is there a safe way to move /var/tmp to /mnt/intflash (/dev/sdb1) instead?
Also, when I started this thread, I thought this would need to be in the PBS forum, but it probably needs to be moved.

 

[bigun89]

I found out what happened. After an update, my vzdump.conf got overwritten and the dump dir I had specified got removed. One of my own old thread clued me in.