Disk Full during Backup, no longer responsive

scrub_nub888

New Member
May 25, 2023
9
1
3
Hi All,

I had an automated backup setup. This backup was sent to a separate PBS node with plenty of storage. For some reason the node storage filled up to 100% and became unresponsive. I'm thinking the backup job filled up the local storage first? I'm not sure how to go about this. I can only access CLI physically on the node.

Any ideas?
 
First of all, check which folder or partition has actually filled up. You can do this with "df" for mountpoints and with "du" for folders. Only then will it be possible to see what has been filled up here.
 
Last edited:
Here's what i'm seeing. I see pve-1 is full but when I go in there there isn't much?

Not sure what I should/shouldn't delete

IMG_5144.jpeg
 
Awesome, that was helpful, I was able to find a file/directory called /var/tmp/vzdumptmp84023_106 that is 1.2T! How can I delete this? I tried rm -rf but that didn't seem to work
Please be careful with rm -rf. It's a temporary file created during a backup, so that makes sense. rm /var/tmp/vzdumptmp84023_106 should work. What error are you getting?
 
Please be careful with rm -rf. It's a temporary file created during a backup, so that makes sense. rm /var/tmp/vzdumptmp84023_106 should work. What error are you getting?
gotcha, i was able to get it deleted - thank you!

How can I prevent backups from filling up this local storage?
 
How can I prevent backups from filling up this local storage?
As the saying goes, a lot helps a lot - at least that's the first option.
Are you actually using that much storage space or is there probably still a lot of free space in the VM? Do you use the discard flag on the VMs?
 
As the saying goes, a lot helps a lot - at least that's the first option.
Are you actually using that much storage space or is there probably still a lot of free space in the VM? Do you use the discard flag on the VMs?
so I have 2 pools an ssd pool and hdd pool.

ssd pool (proxmox is also installed on here) is 1.91TB with 1.32TB Free
hdd pool is 31.99TB with 22.36TB Free

This VM I was backing up was 2.4TB and is on the HDD pool, I guess it used the local drive (on the ssd pool)?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!