Disks Full

4castiel

New Member
May 10, 2023
4
0
1
Hey,
I'm using proxmox in a homelab environment and have been testing things over the past few days, yesterday I setup a cluster with an additional server.

This morning I logged into the webGUI and saw that both of my VM's were offline due to a lack of storage (I have 2x 1.92TB SSD's in ZFS & 4x 600GB Enterprise Drives in RAID0)

I believe the storage has filled up due to me setting the backup locations incorrectly, I rebooted once I saw the error as I thought it was showing the wrong information.

Since rebooting, I've been unable to access the web GUI, the IP no longer shows on the CLI when starting up the server as it normally does.

I thought removing a out of use VM would be the best option, as I could then go in and remove the incorrectly placed backups (I made a mistake in not setting up prune, so it keeps backups indefinitely) but when running pct destroy 101 -force -purge on CLI I get 'Unable to load access control list: Connection Refused'

So I'm a bit stuck currently, I need to remove the backups in order to get things working again but I can't find a way to delete them via command line.

Any help would be massively appreciated.

Thank you :)


The setup is as follows:

Main Server: pve W/ 2 VM's (100 & 101) and two storage pools local (pve) & local-zfs (pve)

2nd Server: thevault W/ 1 VM (102) and two storage pools local (thevault) & tvstorage (thevault)

Running # du -h -d1 -x / gives the following output:

Code:
root@pve:~#  du -h -d1 -x /
512     /mnt
512     /media
25K     /tmp
512     /opt
512     /srv
2.9T    /var
512     /home
47K     /root
1.1G    /usr
3.6M    /etc
73M     /boot
2.9T    /
 
Last edited:
Update:

It looks like the cultrate is
/var/lib/vz/dump

This folder is 2.9TB in size, any ideas how to clean this up?
 
Running /var/lib/vz/dump# ls gives the following output:

Code:
root@pve:/var/lib/vz/dump# ls
vzdump-qemu-100-2023_05_04-14_38_46.log            vzdump-qemu-100-2023_05_05-17_06_44.vma.zst        vzdump-qemu-100-2023_05_09-16_00_01.vma.zst        vzdump-qemu-100-2023_05_09-20_00_02.vma.zst.notes
vzdump-qemu-100-2023_05_04-14_38_46.vma.zst        vzdump-qemu-100-2023_05_05-17_06_44.vma.zst.notes  vzdump-qemu-100-2023_05_09-16_00_01.vma.zst.notes  vzdump-qemu-100-2023_05_09-22_00_03.log
vzdump-qemu-100-2023_05_04-14_38_46.vma.zst.notes  vzdump-qemu-100-2023_05_09-14_29_50.log            vzdump-qemu-100-2023_05_09-18_00_01.log            vzdump-qemu-100-2023_05_09-22_00_03.vma.zst
vzdump-qemu-100-2023_05_05-08_33_52.log            vzdump-qemu-100-2023_05_09-14_45_31.log            vzdump-qemu-100-2023_05_09-18_00_01.vma.zst        vzdump-qemu-100-2023_05_09-22_00_03.vma.zst.notes
vzdump-qemu-100-2023_05_05-08_33_52.vma.zst        vzdump-qemu-100-2023_05_09-14_45_31.vma.zst        vzdump-qemu-100-2023_05_09-18_00_01.vma.zst.notes  vzdump-qemu-100-2023_05_10-00_00_03.tmp
vzdump-qemu-100-2023_05_05-08_33_52.vma.zst.notes  vzdump-qemu-100-2023_05_09-14_45_31.vma.zst.notes  vzdump-qemu-100-2023_05_09-20_00_02.log            vzdump-qemu-100-2023_05_10-00_00_03.vma.dat
vzdump-qemu-100-2023_05_05-17_06_44.log            vzdump-qemu-100-2023_05_09-16_00_01.log            vzdump-qemu-100-2023_05_09-20_00_02.vma.zst
 
Resolved!

Turns out the backups were 100% storing in the wrong place, I deleted my uploaded ISO files and the web interface started working again.

Then logged into the Web UI and deleted the offending backups.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!