ZFS storage disk image using more space then expected

[mschultz123]

I'm hoping someone can help me make sense of my storage usage. I have a single node of Proxmox VE 6.1-7. I have a ZFS raidz1 of 9 4TB disks. after parity, I have 28.18 TiB but when I add a disk image to a VM I can't use anywhere near that much space. the only content on the ZFS storage is a 14.6 TiB disk image but when I look in the summary section it says I'm using 94.03% (26.50 TiB of 28.18 TiB). how is a 14.6 TiB disk image using 26.50 TiB of space? where is all that extra space going? thanks in advance.

 

[budy]

Yeah… well… the storage summary is really not that useful. It only shows you the physical storage without taking the type of FS on that storage into account. I am running ZFS with 4 x 1.8TiB disks in a 2 x 2 mirror pool and the storage summary shows, that I do have 4.42 TiB of 6.46 TiB left, when in fact there's only 1TiB left available on the zpool.

Better go check the ZFS stats on your actual PVE host to get real numbers. Also make sure, that you take all ZFS snapshots into account… they can consume a lot of space, if they're left and never deleted from the pool.

 

[LnxBil]

Also make sure, that you take all ZFS snapshots into account… they can consume a lot of space, if they're left and never deleted from the pool.

Unfortunately, you cannot see how much space is freed after deletion before you doing so.

Yeah… well… the storage summary is really not that useful.

ZFS is different and also the tools itself are different. The CLI tools zfs and zpool report different values, due to the fact that zpool refers lists the vdev space and not the space after redundancy. You can see this here:

root@backup ~ > zpool list rpool
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool  48,8T  32,4T  16,3T         -    14%    66%  1.00x  ONLINE  -

root@backup ~ > zfs list rpool
NAME    USED  AVAIL  REFER  MOUNTPOINT
rpool  21,6T  9,84T   224K  /rpool

Depending on the metadata, snapshots, ashift value and potential refreservation, you can have a lot less space available than you might think.