Whilst move storage from local-lvm to TrueNAS ZFS over iSCSI: Warning: volblocksize (4096) is less than the default minimum block size (8192).

[eugenevdm]

Hi! When I move a VM's storage from `local-lvm` to a TrueNAS Core based ZFS volume over iSCSI, I get the following warning:

Task viewer: VM 102 - Move disk

create full clone of drive scsi0 (local-lvm:vm-102-disk-0)
Warning: volblocksize (4096) is less than the default minimum block size (8192).
To reduce wasted space a volblocksize of 8192 is recommended.

In order to get ZFS over iSCSI working, I used the plugin https://github.com/TheGrandWazoo/freenas-proxmox

I have no idea how to approach this issue. As far as I know I have chosen every single default possible.

Please could someone give me some tips on what the next steps of the troubleshooting process is?

 

[fiona]

Hi,
on the host with ZFS, what does zfs get volblocksize <name of ZFS> say? Please also share your storage configuration /etc/pve/storage.cfg.

 

[eugenevdm]

ZFS host block size configuration for disk I'm moving:

zfs get volblocksize pool/vm-102-disk-0
NAME                PROPERTY      VALUE     SOURCE
pool/vm-102-disk-0  volblocksize  4K        -

PVE Storage configuration:

cat /etc/pve/storage.cfg
dir: local
    path /var/lib/vz
    content vztmpl,backup,iso

lvmthin: local-lvm
    thinpool data
    vgname pve
    content rootdir,images

lvm: new-lvm-volume
    vgname pve
    content images,rootdir
    shared 0

lvmthin: new-lvm-thin-volume
    thinpool data
    vgname pve
    content rootdir,images

zfs: iscsi3
    blocksize 4k
    iscsiprovider freenas
    pool pool
    portal 192.168.1.25
    target iqn.2005-10.org.freenas.ctl:target
    content images
    freenas_apiv4_host 192.168.1.25
    freenas_password secret
    freenas_use_ssl 0
    freenas_user root
    nowritecache 0
    sparse 1

 

[fiona]

zfs: iscsi3
    blocksize 4k

The setting is here. Because of this, new volumes allocated on the storage will get this volblocksize.

 

[eugenevdm]

After battling to understand this problem and all the terminology in my near perfect Proxmox VE environment, I have finally made substantial progress in solving the issue for (my environment).

Environment:

- Proxmox VE
- ZFS over iSCSI to TrueNAS Scale

When you set up ZFS over iSCSI with Proxmox VE the default volblocksize is 4k.

When I started using TrueNAS Core, which most likely relies on OpenZFS, the default volblocksize was 8k.

Somewhere along the line TrueNAS core updated, and therefor as per https://github.com/openzfs/zfs/pull/12406 the ZFS default now became 16k.

So that's why out of the blue when I got new messages, namely:

Warning: volblocksize (8192) is less than the default minimum block size (16384).
To reduce wasted space a volblocksize of 16384 is recommended.

I was really confused. After much reading, and discovering a scary script but with nice explanations that can retrospectively change this value, I had another thought:

What if I just create a new ZFS over iSCSI to the exact same server but using 16k instead? And then what if I then just transfer move the data from NASX to NASX-16k?

So far, so good. Will keep you posted.