Host disk full

[Loic92]

Hello,
I'm using Promox ve 8.
My host disk space is 100% full after a backup vm job tentative.
Now my host is in bad state, I cannot delete the backup (tmp) file via the UI ([VM]\Backup), "Loading" or an empty list is displayed.
The "Tasks" list is indicating that the VM backup job is still ongoing.
I can access to the host by ssh but I don't find any way to delete the files into /var/lib/vz/dump/ by using CLI:

root@pve21:~# ll /var/lib/vz/dump/
total 179805506
drwxr-xr-x 3 root root 4 Aug 16 05:00 .
drwxr-xr-x 7 root root 7 Aug 15 22:16 ..
drwxr-xr-x 2 root root 4 Aug 16 05:00 vzdump-qemu-102-2023_08_16-05_00_56.tmp
-rw-r--r-- 1 root root 183997153280 Aug 16 05:09 vzdump-qemu-102-2023_08_16-05_00_56.vma.dat

What is the solution?
I already tried to reboot the host.

Thanks.

 

[Maximiliano]

Hello, what filesystem are you using? e.g. is this ext4+lvm, zfs, etc?

 

[Loic92]

zfs

 

[Maximiliano]

Ideally, you should never allow a ZFS drive to be close to 100%. To get yourself out of this situation you can delete a file with cat /dev/null > /path/to/big/file. If I am not mistaken that won't trigger a copy-on-write if the disk is at 100%.

 

[Loic92]

it's not working

root@pve21:~# cat /dev/null > /var/lib/vz/dump/vzdump-qemu-102-2023_08_16-05_00_56.vma.dat
-bash: /var/lib/vz/dump/vzdump-qemu-102-2023_08_16-05_00_56.vma.dat: No space left on device

 

[Loic92]

finally I just did:
rm -f /var/lib/vz/dump/vzdump-qemu-102-2023_08_16-05_00_56.vma.dat

My initial goal was to try to cancel the backup job and to remove the (tmp) files properly but apparently the only solution was to force the deletation manually.
Thanks.