My rpool/ROOT/pve-1 filesystem is full

[trilljester]

Hi there,

My ZFS filesystem for / is showing completely used up, but I can't find where it went bad. We had an unexpected power loss the other day right when this system was running backups, so I suspect that might be the cause.

Here's the output from zfs list:

NAME               USED  AVAIL  REFER  MOUNTPOINT
rpool             56.2G     0B   104K  /rpool
rpool/ROOT        56.2G     0B    96K  /rpool/ROOT
rpool/ROOT/pve-1  56.2G     0B  56.2G  /
rpool/data          96K     0B    96K  /rpool/data

Can I get some help on tracking down what's taking up all the available space?

 

[bbgeek17]

du -h -x -d1 /
du -h -x -d1 /var
etc

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[trilljester]

Results from both of those commands:

root@alderaan:/var# du -h -x -d1 /
512    /media
715M    /usr
8.9M    /bin
1.0K    /lib64
3.9M    /etc
8.2M    /sbin
512    /home
357M    /lib
173K    /root
28K    /tmp
1.0K    /mnt
512    /srv
512    /opt
344M    /var
95M    /boot
1.5G    /

root@alderaan:/var# du -h -x -d1 /var
66K    /var/tmp
512    /var/mail
512    /var/opt
251M    /var/cache
550K    /var/spool
21M    /var/log
1.7M    /var/backups
512    /var/local
71M    /var/lib
344M    /var

 

[bbgeek17]

If the outputs from the first and 3rd post are from the same server, then it doesnt make much sense.

There are a quiet a few articles online on how troubleshoot ZFS space, start running through commands to see if you can find smoking gun:
https://oshogbo.vexillium.org/blog/65/
https://docs.oracle.com/cd/E37838_01/html/E61000/tsgeneral-10.html
https://docs.oracle.com/cd/E36784_01/html/E36835/gbbti.html
etc

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox