ZFS wrong used space

pauet

New Member
Jan 22, 2019
8
0
1
24
I have a raid1 pool for storage VM Disks, but it show a 99,99% used space when I should have some free space.

data/enc 1.76T 262M 192K /data/enc
data/enc/vm-104-disk-0 625G 558G 66.8G -
data/enc/vm-105-disk-0 1.15T 722G 450G -


The VM-104-disk-0 is set to 600GB and VM-105-disk-0 is set to 700GB, this would be 1300GB. There aren't any snapshot.

Is there any problem running a pool with a 99,99% used space? Why there are no free space? And why if I've set a 700GB Raw disk image the Zfs says it's 1.15TB?

Thanks
 
After a few days running everything seems working, but I wonder why is this happening and why I'm unable to create another virtual disk at this storage and make snapshots (I didn't try to create a snapshot).
 
you need to show more information to determine the cause of your space usage. e.g.,
Code:
zpool status -v
zpool list -v
zpool get ashift
zfs list -space,compressratio,recordsize,volblocksize -r -t all

in general, you should ensure that your zpools don't get full or even near-full. it's a copy on write filesystem, and if there is no space to quickly allocate for copies, performance will suffer massively.
 
Bash:
root@kenia:~# zpool status -v
  pool: data
state: ONLINE
  scan: scrub repaired 0B in 0 days 00:00:01 with 0 errors on Sun Nov 10 00:24:02 2019
config:

    NAME                                 STATE     READ WRITE CKSUM
    data                                 ONLINE       0     0     0
      mirror-0                           ONLINE       0     0     0
        ata-ST2000DM006-2DM164_Z4Z8VJAZ  ONLINE       0     0     0
        ata-ST2000DM006-2DM164_Z8E0Q8N8  ONLINE       0     0     0

errors: No known data errors

Bash:
root@kenia:~# zpool list -v
NAME                                                    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data                                                   1.81T   971G   885G        -         -     1%    52%  1.00x    ONLINE  -
  mirror                                               1.81T   971G   885G        -         -     1%  52.3%      -  ONLINE 
    ata-ST2000DM006-2DM164_Z4Z8VJAZ                        -      -      -        -         -      -      -      -  ONLINE 
    ata-ST2000DM006-2DM164_Z8E0Q8N8                        -      -      -        -         -      -      -      -  ONLINE
Bash:
root@kenia:~# zpool get ashift
NAME   PROPERTY  VALUE   SOURCE
data   ashift    0       default
 
Bash:
root@kenia:~# zfs list -o space,compressratio,recordsize,volblocksize -r -t all
NAME                                                             AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  RATIO  RECSIZE  VOLBLOCK
data                                                              241M  1.76T        0B     96K             0B      1.76T  1.00x     128K         -
data@zfs-auto-snap_hourly-2019-11-10-1217                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1317                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1417                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1517                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1617                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1717                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1817                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-1917                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-2017                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-2117                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-2217                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-10-2317                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0017                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0117                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0217                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0317                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0417                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0517                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_daily-2019-11-11-0525                             -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0617                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_hourly-2019-11-11-0717                            -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_frequent-2019-11-11-0745                          -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_frequent-2019-11-11-0800                          -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_frequent-2019-11-11-0830                          -     0B         -       -              -          -  1.00x        -         -
data@zfs-auto-snap_frequent-2019-11-11-0845                          -     0B         -       -              -          -  1.00x        -         -
data/enc                                                          241M  1.76T        0B    192K             0B      1.76T  1.00x     128K         -
data/enc@zfs-auto-snap_daily-2019-11-10-1131                         -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_daily-2019-11-10-1132                         -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1217                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1317                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1417                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1517                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1617                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1717                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1817                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-1917                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-2017                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-2117                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-2217                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-10-2317                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0017                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0117                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0217                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0317                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0417                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0517                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_daily-2019-11-11-0525                         -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0617                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_hourly-2019-11-11-0717                        -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_frequent-2019-11-11-0745                      -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_frequent-2019-11-11-0800                      -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_frequent-2019-11-11-0830                      -     0B         -       -              -          -  1.00x        -         -
data/enc@zfs-auto-snap_frequent-2019-11-11-0845                      -     0B         -       -              -          -  1.00x        -         -
 
and the other part (messages limited to 1000)

Bash:
root@kenia:~# zfs list -o space,compressratio,recordsize,volblocksize -r -t all
NAME                                                             AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  RATIO  RECSIZE  VOLBLOCK
data                                                              241M  1.76T        0B     96K             0B      1.76T  1.00x     128K         -

data/enc/vm-104-disk-0                                            109G   625G      773M    515G           109G         0B  1.00x        -        8K
data/enc/vm-104-disk-0@zfs-auto-snap_frequent-2019-11-11-0845        -   773M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0                                            718G  1.15T     3.28G    452G           717G         0B  1.00x        -        8K
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-1617          -   178M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-1717          -  38.3M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-1817          -  6.17M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-1917          -  35.6M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-2017          -   113M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-2117          -  1.41M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-2217          -  1.38M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-10-2317          -  18.1M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0017          -  6.42M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0117          -  6.80M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0217          -  79.6M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0317          -  9.48M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0417          -     0B         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0517          -     0B         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_daily-2019-11-11-0525           -     0B         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0617          -     0B         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_hourly-2019-11-11-0717          -  2.16M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_frequent-2019-11-11-0745        -  2.23M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_frequent-2019-11-11-0800        -  1.53M         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_frequent-2019-11-11-0830        -     8K         -       -              -          -  1.00x        -         -
data/enc/vm-105-disk-0@zfs-auto-snap_frequent-2019-11-11-0845        -     8K         -       -              -          -  1.00x        -         -
 
you are using zvols with reservation and snapshots, so your zvols will always take up their full size + any data only referenced by snapshots. e.g., vm-105-disk-0 seems to currently take up:
- 3G for data only referenced by one or more snapshots
- 452G for data currently referenced by the dataset (and potentially also by one or more snapshots)
- 717G for reserved space (needed to allow you to successfully overwrite the whole zvol)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!