Proxmox storage different than ZFS usage?

[jlficken]

I'm confused as I have a local ZFS pool with 4 x 8TB HDD's (2 x 2 drive mirrors) and I'm seeing conflicting usage.

If I select the local storage node i the Server View it's saying under the Summary tab that 11.91TB of 15.84TB is used which is 75.21%, however, if I go to the Storage tab when selecting the node and click on the ZFS tab it says that 4.23TB of 15.98TB is allocated with 0% fragmentation.

If I go look at the VM Disks tab of the pool under the Server View for the nodes I see ~7TB of drives exist on this pool for various VM's. There are no CT Volumes on this pool.

This same configuration is set up on 2 different nodes and they both report the same discrepancy.

What do I believe?

 

[Dunuin]

I would recommend you look at the CLI to see what's going on:
zpool list -v
zfs list -o space

 

[jlficken]

I actually just found the zfs list command after more research.

I've posted the results below and will keep researching as I don't quite grasp what I'm looking at currently.

zpool list -v:

NAME                         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
VMs_Data                    14.5T  3.85T  10.7T        -         -     0%    26%  1.00x    ONLINE  -
  mirror-0                  7.27T  1.87T  5.40T        -         -     0%  25.7%      -    ONLINE
    scsi-35000cca23b063bf4  7.28T      -      -        -         -      -      -      -    ONLINE
    scsi-35000cca260ca5108  7.28T      -      -        -         -      -      -      -    ONLINE
  mirror-1                  7.27T  1.98T  5.28T        -         -     0%  27.3%      -    ONLINE
    scsi-35000cca260c9fdd8  7.28T      -      -        -         -      -      -      -    ONLINE
    scsi-35000cca260ca4e88  7.28T      -      -        -         -      -      -      -    ONLINE

zfs list -o space:

NAME                    AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
VMs_Data                3.57T  10.8T        0B     96K             0B      10.8T
VMs_Data/vm-101-disk-0  7.63T  7.06T      512K   2.99T          4.06T         0B
VMs_Data/vm-104-disk-0  3.67T   136G        0B   34.0G           102G         0B
VMs_Data/vm-105-disk-0  4.59T  1.05T        0B   38.1G          1.02T         0B
VMs_Data/vm-107-disk-0  3.82T   326G      368K   72.3G           254G         0B
VMs_Data/vm-110-disk-0  3.67T   102G        0B     56K           102G         0B
VMs_Data/vm-112-disk-0  3.87T   445G        0B    140G           305G         0B
VMs_Data/vm-113-disk-0  3.72T   215G        0B   63.0G           152G         0B
VMs_Data/vm-113-disk-1  4.59T  1.53T        0B    528G          1.02T         0B

 

[Dunuin]

My guess would be that you either forgot to check the "Thin" checkbox when creating the ZFS storage. So all virtual disks will be thick-provisioned. Or you never cared about setting up discard for the virtual disks and inside every guestOS.

 

[jlficken]

I got it figured out now I believe with help from these posts:

https://www.reddit.com/r/zfs/comments/kso5qt/high_usedrefreserv_on_proxmox/
https://forum.proxmox.com/threads/zfs-enable-thin-provisioning.41549/

 

[jlficken]

My guess would be that you either forgot to check the "Thin" checkbox when creating the ZFS storage. So all virtual disks will be thick-provisioned. Or you never cared about setting up discard for the virtual disks and inside every guestOS.

That's a bingo on the "Thin Provision" checkbox guess. I'm still relatively new to Proxmox as I've only had my cluster up for about 18 months now and didn't realize that was a thing.

I need to read more and figure out in what circumstances I *wouldn't* want to check that box.

For the time being I've checked it on and will go fix the disks that were created before it was checked.

Thanks!!!

 

[jlficken]

Running this on each volume for each node after check on "Thin provision" for each pool has me squared away...thanks!!

zfs set refreservation=none VMs/vm-114-disk-0