Backup failed - No space left on device

[tukane]

Hi all,

since a few days my backup job is failing.

ERROR: rsync: [receiver] write failed on "/var/tmp/vzdumptmp2094692_102/var/lib/docker/containers/8bb2ef4ed71fc6d6e114de76491a145d917d3369c422f4f97d549d52aac22057/8bb2ef4ed71fc6d6e114de76491a145d917d3369c422f4f97d549d52aac22057-json.log": No space left on device (28)
ERROR: rsync error: error in file IO (code 11) at receiver.c(381) [receiver=3.2.7]
ERROR: rsync: [sender] write error: Broken pipe (32)

The backups should be stored on my NAS.
From the error message above I assume that the backup file is stored in a temp directory and copied from there to the NAS, right?

I'm running 1 container (64G) and 1 VM (64G). Proxmox is running on a 256G SSD, so when my assumption with the tmp-file is right, the disk size is not sufficient.

root@proxmox:/usr# df -h
Filesystem                     Size  Used Avail Use% Mounted on
udev                           7.8G     0  7.8G   0% /dev
tmpfs                          1.6G  2.2M  1.6G   1% /run
/dev/mapper/pve-root            67G  9.6G   54G  16% /
tmpfs                          7.8G   40M  7.8G   1% /dev/shm
tmpfs                          5.0M     0  5.0M   0% /run/lock
/dev/sda2                     1022M   12M 1011M   2% /boot/efi
//192.168.0.10/Backup/proxmox   14T  9.9T  4.1T  71% /mnt/pve/NAS
//192.168.0.10/paperless        14T  9.9T  4.1T  71% /mnt/lxc_shares/paperless
tmpfs                          1.6G     0  1.6G   0% /run/user/0
/dev/fuse                      128M   20K  128M   1% /etc/pve

I've another HDD (sdb) in my proxmox server which has more capacity.


On this HDD is the volume of the container stored.


Is there a way to use this HDD for the tmp-files of the backups? If yes, what do I have to consider? Do I need another partition? What about the CT Volume?

Thanks
tukane

 

[gfngfn256]

Is there a way to use this HDD for the tmp-files of the backups? If yes, what do I have to consider? Do I need another partition?

Create a directory storage on that sdb drive & then set a tmpdir for vzdump on that location path.
I see you have LVM on that drive already. So AFAIK you will need to create a new LV to be able to accomplish this.

See my post here which references the docs on this. With a NAS as the target, you should probably anyway have had this.

 

[tukane]

Thanks @gfngfn256 for your reply.
I thought about another solution. I moved all disks to the second drive so that local-lvm has no data on it.
When I delete the local-lvm storage is there a way to extend the local storage without loosing the data or reinstalling proxmox?

 

[gfngfn256]

When I delete the local-lvm storage is there a way to extend the local storage without loosing the data or reinstalling proxmox?

Yes you can. You can use the the lvresize or lvextend command to do this. You'll then need to resize the fs with resize2fs .

Here is a possible sample of that procedure, to remove the whole local-lvm (thin) & then regain ALL of that space for the root directory.

lvremove /dev/pve/data
lvresize -l +100%FREE /dev/pve/root
resize2fs /dev/mapper/pve-root

You may want to leave some space to recreate a smaller local-lvm (thin) - and adjust the above accordingly.
If you do decide to recreate a smaller local-lvm (thin) - you will have to subsequently create a new LV, convert it to lvm-thin & then add it as a storage in PVE.

Whatever you do: MAKE SURE YOU HAVE FULLY RESTORABLE BACKUPS OF ALL YOUR DATA!