VM /mnt Data Erroneously Landing on rpool

Benneffect

New Member
Mar 21, 2025
8
1
3
My PVE rpool has ghost data that I cannot remove.

I misconfigured VirtIOFS storage and somehow got rpool to contain my /mnt/vmdata rather than it being located on the intended storage pool, a 12TB RAIDZ pool (16TB raw). The storage was shared to my Ubuntu Server VM. Upon writing data (Immich, Nextcloud), my rpool filled up to over 100% and my VM stopped booting.

I've gone through troubleshooting, and I think I have properly set the storage location the the correct pool. But, I cannot delete the data that is accidentally in the rpool. I've deleted the data off the rpool, but it still shows as mostly full. And, when I start writing data to the VirtIOFS is still starts filling up again.

This is my first hypervisor & self-hosting project that I am slowly working on, so I am quite newb-ish in PVE / Ubuntu / Linux - I've been using round-robin help from AI, but they are not helping me solve my error. What files do I need to check to ensure my "/mnt/Data" is located on the right pool? How do I get the data off the rpool?

Here is what my VM sees:
ben@cloud357:/mnt$ tree -d
.
└── cloud357-data
├── immich-photos
├── personal-files [error opening dir]
└── template
└── cache

6 directories


Here is what the host has:
root@pve1:/mnt# tree -d
.
└── Data
└── cloud357-data
├── immich-photos
├── personal-files
│ └── nextcloud-data
└── template
└── cache
 
Hi, @Benneffect
Probably your intended pool wasn't mounted at /mnt
So the data landed on the "/" filesystem (which contains /mnt directory).

The fix is to umount whatever is mounted at /mnt (first shutdown VMs if they use this share).
Then inspect what really is inside /mnt

If /mnt contents is also in other secure place, you can delete the contents in /mnt
This should release "/" filesystem usage.
 
The problem isn't that the my directories aren't in the correct location - I've fixed that (I think). My problem is that rpool thinks it is full when I cannot find anything there.

root@pve1:/# ls
bin Cloud357-Boot-Mirror dev home lib64 mnt proc rpool sbin sys usr Win-Lab-Mirror
boot data etc lib media opt root run srv tmp var

root@pve1:/# zfs list
NAME USED AVAIL REFER MOUNTPOINT
Cloud357-Boot-Mirror 861G 38.4G 96K /Cloud357-Boot-Mirror
Cloud357-Boot-Mirror/vm-232-disk-0 3M 38.4G 120K -
Cloud357-Boot-Mirror/vm-232-disk-1 861G 639G 260G -
Data 187G 10.3T 200K /mnt/Data
Data/cloud357-data 187G 10.3T 187G /mnt/Data/cloud357-data
Win-Lab-Mirror 2.23M 108G 96K /Win-Lab-Mirror
rpool 195G 17.9G 104K /rpool
rpool/ROOT 195G 17.9G 96K /rpool/ROOT
rpool/ROOT/pve-1 195G 17.9G 195G /
rpool/data 96K 17.9G 96K /rpool/data
rpool/mnt 104K 17.9G 104K /mnt
rpool/var-lib-vz 120K 17.9G 120K /var/lib/vz

root@pve1:/# zfs get mountpoint Data
NAME PROPERTY VALUE SOURCE
Data mountpoint /mnt/Data local

root@pve1:/# zfs get mountpoint Data/cloud357-data
NAME PROPERTY VALUE SOURCE
Data/cloud357-data mountpoint /mnt/Data/cloud357-data inherited from Data

root@pve1:/# zfs list -r rpool/mnt
NAME USED AVAIL REFER MOUNTPOINT
rpool/mnt 104K 17.9G 104K /mnt

...

root@pve1:/rpool/ROOT/pve-1# ls

root@pve1:/rpool/ROOT/pve-1# du -sh --apparent-size
0

root@pve1:/# zfs get mountpoint Data
NAME PROPERTY VALUE SOURCE
Data mountpoint /mnt/Data local

root@pve1:/# zfs get mountpoint Data/cloud357-data
NAME PROPERTY VALUE SOURCE
Data/cloud357-data mountpoint /mnt/Data/cloud357-data inherited from Data

root@pve1:/# zfs get mountpoint rpool/mnt
NAME PROPERTY VALUE SOURCE
rpool/mnt mountpoint /mnt local
 
Last edited: