[SOLVED] PBS Root Disk Full can't sync, backup or run garbage collection.

Jarvar

Member
Aug 27, 2019
204
8
23
A few days ago, my Proxmox Backup Server filled up in the root folder. I managed to delete some large log files then and get functional.
Last night during a remote sync it filled up and my job failed.
Could somebody please help me diagnose the issue?
Thanks, This is out of a 100 GB drive.

this is what I found so far.
Code:
root@pbs1:~# du -sh /* | sort -hr
du: cannot access '/proc/1177/task/1177/fd/4': No such file or directory
du: cannot access '/proc/1177/task/1177/fdinfo/4': No such file or directory
du: cannot access '/proc/1177/fd/3': No such file or directory
du: cannot access '/proc/1177/fdinfo/3': No such file or directory
1.7T    /backup
3.9G    /usr
1.1G    /var
574M    /boot
4.0M    /etc
740K    /run
80K     /root
40K     /tmp
16K     /lost+found
12K     /mnt
4.0K    /srv
4.0K    /opt
4.0K    /media
4.0K    /home
4.0K    /disk3
0       /sys
0       /sbin
0       /proc
0       /libx32
0       /lib64
0       /lib32
0       /lib
0       /dev
0       /bin
root@pbs1:~#
 
Last edited:

datschlatscher

Proxmox Staff Member
Staff member
Mar 14, 2022
131
42
28
A bit more information about your storage topology is needed here. Do you have any hidden folders in your root directory for some reason? du -sh /.* | sort -hr

Is /backup mounted directly or do you have subfolders mounted in there? What does lsblk say?

You should be able to get a hold of where most of your data by using something like baobab (Disk Usage Analyzer) if you are running a Desktop environment. If you only have terminal access consider a tool like ncdu. Both being really easy to use and great to quickly tell where most of your disk space goes.
 

Jarvar

Member
Aug 27, 2019
204
8
23
root@pbs1:/backup# du -sh /.* | sort -hr du: cannot access '/./proc/1298/task/1298/fd/4': No such file or directory du: cannot access '/./proc/1298/task/1298/fdinfo/4': No such file or directory du: cannot access '/./proc/1298/fd/3': No such file or directory du: cannot access '/./proc/1298/fdinfo/3': No such file or directory 1.9T /. root@pbs1:/backup#

I didn't do anything specific to hide anything in the root directory.
/backup was mounted to an external USB drive for the datastore.

At this moment I don't have enough free space to install ncdu\

lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 100G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 512M 0 part └─sda3 8:3 0 99.5G 0 part ├─pbs-swap 253:0 0 3.9G 0 lvm [SWAP] └─pbs-root 253:1 0 95.6G 0 lvm / sdb 8:16 0 3.6T 0 disk └─sdb1 8:17 0 3.6T 0 part /mnt/usb-drive01/disk04 sr0 11:0 1 1024M 0 rom root@pbs1:/backup#
 

Jarvar

Member
Aug 27, 2019
204
8
23
I think I solved my own issue. There was time where something didn't mount correctly. I mounted an NFS as well to backup the contents of my external hard drive to rsync things over as an added backup. When it failed, it filled up.
So I edited it out in the fstab, rebooted, deleted the contents of the folder which was about 89G and I'm cleaning up old mounts and will try it again.
df -h brings up only 7% drive use now which is good for me now.
I'll apt install ncdu -y now.
Thank you so much!
 
  • Like
Reactions: datschlatscher

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!