zfs error: cannot create snapshot: out of space

krystianen · Jun 6, 2022

Hi,

I cant create snapshot on PVE. it wants more space but i think have enougth.

Can u help please?

Code:

saving VM state and RAM using storage 'local-1tb'
4.01 MiB in 0s
1.20 GiB in 1s
1.67 GiB in 2s
2.07 GiB in 3s
2.50 GiB in 4s
2.69 GiB in 5s
2.69 GiB in 6s
2.69 GiB in 7s
2.69 GiB in 8s
2.69 GiB in 9s
2.69 GiB in 10s
2.69 GiB in 11s
2.69 GiB in 12s
2.69 GiB in 13s
2.69 GiB in 14s
2.69 GiB in 15s
2.69 GiB in 16s
2.69 GiB in 17s
2.69 GiB in 18s
2.69 GiB in 19s
3.30 GiB in 20s
3.36 GiB in 21s
3.36 GiB in 22s
3.36 GiB in 23s
3.36 GiB in 24s
3.36 GiB in 25s
3.36 GiB in 26s
3.36 GiB in 27s
3.36 GiB in 28s
3.36 GiB in 29s
3.36 GiB in 30s
3.36 GiB in 31s
3.36 GiB in 32s
3.36 GiB in 33s
3.36 GiB in 34s
3.36 GiB in 35s
3.36 GiB in 36s
3.36 GiB in 37s
3.36 GiB in 38s
3.36 GiB in 39s
3.36 GiB in 40s
3.36 GiB in 41s
3.36 GiB in 42s
3.36 GiB in 43s
3.36 GiB in 44s
3.36 GiB in 45s
3.36 GiB in 46s
3.36 GiB in 47s
3.84 GiB in 48s
4.28 GiB in 49s
4.69 GiB in 50s
4.77 GiB in 51s
4.81 GiB in 52s
4.86 GiB in 53s
4.93 GiB in 54s
4.93 GiB in 55s
4.93 GiB in 56s
4.93 GiB in 57s
4.93 GiB in 58s
4.93 GiB in 59s
reducing reporting rate to every 10s
5.65 GiB in 1m 9s
5.65 GiB in 1m 19s
7.06 GiB in 1m 29s
7.84 GiB in 1m 39s
7.85 GiB in 1m 49s
7.85 GiB in 2m
9.80 GiB in 2m 10s
10.42 GiB in 2m 20s
10.42 GiB in 2m 30s
completed saving the VM state in 2m 39s, saved 11.55 GiB
snapshotting 'drive-scsi0' (local-zfs:vm-100-disk-0)
snapshotting 'drive-scsi1' (local-zfs:vm-100-disk-1)
snapshotting 'drive-scsi2' (local-zfs:vm-100-disk-2)
snapshotting 'drive-scsi3' (local-1tb:vm-100-disk-0)
snapshot create failed: starting cleanup
TASK ERROR: zfs error: cannot create snapshot 'local-1tb/vm-100-disk-0@s005': out of space

Code:

root@BRGKDC1:~# zfs list -t all
NAME                              USED  AVAIL     REFER  MOUNTPOINT
local-1tb                         696G   203G       96K  /local-1tb
local-1tb/vm-100-disk-0           670G   286G      588G  -
local-1tb/vm-100-disk-0@s001     84.9M      -      106M  -
local-1tb/vm-100-state-s001      25.3G   225G     3.19G  -
rpool                             176G   255G       96K  /rpool
rpool/ROOT                       1.11G   255G       96K  /rpool/ROOT
rpool/ROOT/pve-1                 1.11G   255G     1.11G  /
rpool/data                        174G   255G       96K  /rpool/data
rpool/data/vm-100-disk-0         76.2G   255G     41.5G  -
rpool/data/vm-100-disk-0@ALL     12.8G      -     32.8G  -
rpool/data/vm-100-disk-0@s001    14.3G      -     35.2G  -
rpool/data/vm-100-disk-1          530M   255G      530M  -
rpool/data/vm-100-disk-1@ALL        8K      -      530M  -
rpool/data/vm-100-disk-1@s001       0B      -      530M  -
rpool/data/vm-100-disk-2          108K   255G      108K  -
rpool/data/vm-100-disk-2@ALL        0B      -      108K  -
rpool/data/vm-100-disk-2@s001       0B      -      108K  -
rpool/data/vm-101-disk-0         91.9G   255G     72.4G  -
rpool/data/vm-101-disk-0@Gtowe    383M      -     23.2G  -
rpool/data/vm-101-disk-0@Trzeci   404M      -     23.2G  -
rpool/data/vm-101-state-Gtowe    2.77G   255G     2.77G  -
rpool/data/vm-101-state-Trzeci   2.80G   255G     2.80G  -
root@BRGKDC1:~#

krystianen · Jun 6, 2022

any one can help pls?

dcsapak · Jun 7, 2022

did you enable thin provisioning on the storage 'local-1tb' ?
if not, the snapshot takes as much space as the volume itself, and it uses 670G already (only 286G available)

fabian · Jun 7, 2022

https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_zfs_raid_considerations

krystianen · Jun 7, 2022

dcsapak said:
did you enable thin provisioning on the storage 'local-1tb' ?
if not, the snapshot takes as much space as the volume itself, and it uses 670G already (only 286G available)

no i didnt. Can do this during vm works?

krystianen · Jun 7, 2022

dcsapak said:
did you enable thin provisioning on the storage 'local-1tb' ?
if not, the snapshot takes as much space as the volume itself, and it uses 670G already (only 286G available)

i have add check box, but still - out of storage

dcsapak · Jun 8, 2022

the checkbox is only valid for new volumes.
to understand: if the volume is not 'thin provisioned' we set the 'refreservation' to the volume size (so that the storage cannot get accidentally full)
if you do a snapshot now, the snapshot uses the size of the data until now, and the zvol needs again the size (of the refreservation). in
your case there is not enough space for that

you can verify that with

Code:

zfs get all local-1tb/vm-100-disk-0

you can remove the referservation with

Code:

zfs set refreservation=none local-1tb/vm-100-disk-0

but WARNING: if you then overwrite enough data of your disk, the pool might run full and that can have bad consequences (e.g. the vm stops, pool will be exteremely slow, etc)

krystianen · Jun 8, 2022

dcsapak said:
zfs get all local-1tb/vm-100-disk-0

i think i have no refreservation

Code:

root@BRGKDC1:~# zfs get all local-1tb/vm-100-disk-0
NAME                     PROPERTY              VALUE                  SOURCE
local-1tb/vm-100-disk-0  type                  volume                 -
local-1tb/vm-100-disk-0  creation              Wed Jul  7 19:48 2021  -
local-1tb/vm-100-disk-0  used                  670G                   -
local-1tb/vm-100-disk-0  available             311G                   -
local-1tb/vm-100-disk-0  referenced            588G                   -
local-1tb/vm-100-disk-0  compressratio         1.00x                  -
local-1tb/vm-100-disk-0  reservation           none                   default
local-1tb/vm-100-disk-0  volsize               650G                   local
local-1tb/vm-100-disk-0  volblocksize          8K                     default
local-1tb/vm-100-disk-0  checksum              on                     default
local-1tb/vm-100-disk-0  compression           off                    default
local-1tb/vm-100-disk-0  readonly              off                    default
local-1tb/vm-100-disk-0  createtxg             30434                  -
local-1tb/vm-100-disk-0  copies                1                      default
local-1tb/vm-100-disk-0  refreservation        670G                   local
local-1tb/vm-100-disk-0  guid                  12561462539416091158   -
local-1tb/vm-100-disk-0  primarycache          all                    default
local-1tb/vm-100-disk-0  secondarycache        all                    default
local-1tb/vm-100-disk-0  usedbysnapshots       0B                     -
local-1tb/vm-100-disk-0  usedbydataset         588G                   -
local-1tb/vm-100-disk-0  usedbychildren        0B                     -
local-1tb/vm-100-disk-0  usedbyrefreservation  82.3G                  -
local-1tb/vm-100-disk-0  logbias               latency                default
local-1tb/vm-100-disk-0  objsetid              399                    -
local-1tb/vm-100-disk-0  dedup                 off                    default
local-1tb/vm-100-disk-0  mlslabel              none                   default
local-1tb/vm-100-disk-0  sync                  standard               default
local-1tb/vm-100-disk-0  refcompressratio      1.00x                  -
local-1tb/vm-100-disk-0  written               588G                   -
local-1tb/vm-100-disk-0  logicalused           585G                   -
local-1tb/vm-100-disk-0  logicalreferenced     585G                   -
local-1tb/vm-100-disk-0  volmode               default                default
local-1tb/vm-100-disk-0  snapshot_limit        none                   default
local-1tb/vm-100-disk-0  snapshot_count        none                   default
local-1tb/vm-100-disk-0  snapdev               hidden                 default
local-1tb/vm-100-disk-0  context               none                   default
local-1tb/vm-100-disk-0  fscontext             none                   default
local-1tb/vm-100-disk-0  defcontext            none                   default
local-1tb/vm-100-disk-0  rootcontext           none                   default
local-1tb/vm-100-disk-0  redundant_metadata    all                    default
local-1tb/vm-100-disk-0  encryption            off                    default
local-1tb/vm-100-disk-0  keylocation           none                   default
local-1tb/vm-100-disk-0  keyformat             none                   default
local-1tb/vm-100-disk-0  pbkdf2iters           0                      default

dcsapak · Jun 8, 2022

yes you do:

krystianen said:
local-1tb/vm-100-disk-0 refreservation 670G local

670G

Search

Search

zfs error: cannot create snapshot: out of space

krystianen

New Member

krystianen

New Member

dcsapak

Proxmox Staff Member

fabian

Proxmox Staff Member

krystianen

New Member

krystianen

New Member

dcsapak

Proxmox Staff Member

krystianen

New Member

dcsapak

Proxmox Staff Member