Hello,
I have some trouble with my ZFS storage. My VM disks are growing higher than the size that is set in the VM hardware tab,
it results that the host crashes with a full ZFS zpool.
I tried to create a simple example with a 20GiB disk. I think it has to do with the 4k/8k/512 block size, but I cant get the QEMU HARDDISK to use 4k.
Here some debug from VM and Host:
#On VM:
root@vm:/mnt# fdisk -l /dev/sdc
root@vm:/mnt# cryptsetup luksDump /dev/sdc1 | grep 'sector\|area\|discard'
root@vm:/mnt# tune2fs -l /dev/mapper/cryptbackup
root@vm:/mnt# blockdev --getbsz /dev/mapper/cryptbackup
root@vm:/mnt# df -h | grep cryptbackup
#On Host:
root@host:~# zfs get used,available,referenced,referenced,compressratio,volsize,written,logicalused,logicalreferenced,volblocksize rpool/data/vm-
100-disk-2
root@host:~# fdisk -l /dev/zd32
In the example the VM is using 18.6GiB but on the real Storage 27.2GiB are used. I tried it with different file systems all results in nearly results, also without luks. Discard is activated and works. (When files where deleted the used ZFS storage shrinks.)
I think I should get the 4k blocksize also in the VM, but the QEMU HARDDISK stills at 512. Have anyone a tip for me?
Thanks. (Sorry: Edit, wrong VM disk..)
Next Edit - additional information:
Now I have tested it with different filesystems XFS and EXT4 (without luks) and different blocksizes on a zfs volume for a qemu harddisk. Everytime when I copied 8742MB to the file system the zfs volume grows up to 12/13GiB.
Test Results:
xfs 512 on VM Disk
ext4 4096 on VM Disk
ZFS on VM Disk
Why the volume grows up over there real size?
I do not understand this, a physical harddisk can't grow up?
Find a solution: ?
I find out that the same data on a ZFS mounted filesystem do not have this problem:
ZFS filesystem direct (same 8742MB)
So i have passed a host folder with virtio-9p into the VM:
Add args to <vmid>.conf:
fstab in VM:
Result after same data write over VM:
ZFS filesystem over virtio-9p
I have some trouble with my ZFS storage. My VM disks are growing higher than the size that is set in the VM hardware tab,
it results that the host crashes with a full ZFS zpool.
I tried to create a simple example with a 20GiB disk. I think it has to do with the 4k/8k/512 block size, but I cant get the QEMU HARDDISK to use 4k.
Here some debug from VM and Host:
#On VM:
root@vm:/mnt# fdisk -l /dev/sdc
Code:
Disk /dev/sdc: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 12EF217C-A9D4-0346-AC77-7D690772F175
Device Start End Sectors Size Type
/dev/sdc1 2048 41943006 41940959 20G Linux filesystemroot@vm:/mnt# cryptsetup luksDump /dev/sdc1 | grep 'sector\|area\|discard'
Code:
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes]
sector: 512 [bytes]root@vm:/mnt# tune2fs -l /dev/mapper/cryptbackup
Code:
tune2fs 1.44.5 (15-Dec-2018)
tune2fs: Bad magic number in super-block while trying to open /dev/mapper/cryptbackup
/dev/mapper/cryptbackup contains a xfs file systemroot@vm:/mnt# blockdev --getbsz /dev/mapper/cryptbackup
Code:
512root@vm:/mnt# df -h | grep cryptbackup
Code:
/dev/mapper/cryptbackup 20G 19G 1.6G 93% /mnt/cryptbackup#On Host:
root@host:~# zfs get used,available,referenced,referenced,compressratio,volsize,written,logicalused,logicalreferenced,volblocksize rpool/data/vm-
100-disk-2
Code:
NAME PROPERTY VALUE SOURCE
rpool/data/vm-100-disk-2 used 27.2G -
rpool/data/vm-100-disk-2 available 5.90T -
rpool/data/vm-100-disk-2 referenced 27.2G -
rpool/data/vm-100-disk-2 referenced 27.2G -
rpool/data/vm-100-disk-2 compressratio 1.00x -
rpool/data/vm-100-disk-2 volsize 20G local
rpool/data/vm-100-disk-2 written 27.2G -
rpool/data/vm-100-disk-2 logicalused 18.6G -
rpool/data/vm-100-disk-2 logicalreferenced 18.6G -
rpool/data/vm-100-disk-2 volblocksize 4K -root@host:~# fdisk -l /dev/zd32
Code:
Disk /dev/zd32: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 12EF217C-A9D4-0346-AC77-7D690772F175
Device Start End Sectors Size Type
/dev/zd32p1 2048 41943006 41940959 20G Linux filesystemIn the example the VM is using 18.6GiB but on the real Storage 27.2GiB are used. I tried it with different file systems all results in nearly results, also without luks. Discard is activated and works. (When files where deleted the used ZFS storage shrinks.)
I think I should get the 4k blocksize also in the VM, but the QEMU HARDDISK stills at 512. Have anyone a tip for me?
Thanks. (Sorry: Edit, wrong VM disk..)
Next Edit - additional information:
Now I have tested it with different filesystems XFS and EXT4 (without luks) and different blocksizes on a zfs volume for a qemu harddisk. Everytime when I copied 8742MB to the file system the zfs volume grows up to 12/13GiB.
Test Results:
xfs 512 on VM Disk
Code:
/dev/zd48p1 20G 8.7G 12G 44% /mnt/cryptbackup
rpool/data/vm-100-disk-3 12.6G 5.89T 12.6G -ext4 4096 on VM Disk
Code:
/dev/zd48p1 20G 8.6G 10G 47% /mnt/cryptbackup
rpool/data/vm-100-disk-3 12.6G 5.89T 12.6G -ZFS on VM Disk
Code:
tank/text 97G 8.9G 88G 10% /mnt/test
rpool/data/vm-100-disk-4 12.7G 5.87T 12.7G -Why the volume grows up over there real size?
I do not understand this, a physical harddisk can't grow up?
Find a solution: ?
I find out that the same data on a ZFS mounted filesystem do not have this problem:
ZFS filesystem direct (same 8742MB)
Code:
rpool/test 5.9T 8.7G 5.9T 1% /mnt/test
rpool/test 8.60G 5.88T 8.60G /mnt/testSo i have passed a host folder with virtio-9p into the VM:
zfs create -o mountpoint=/mnt/hostshare -o encryption=on -o rpool/hostshareAdd args to <vmid>.conf:
args: -fsdev local,security_model=passthrough,id=fsdev0,path=/mnt/hostshare -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=hostsharefstab in VM:
hostshare /mnt/hostshare 9p trans=virtio,version=9p2000.L 0 1Result after same data write over VM:
ZFS filesystem over virtio-9p
Code:
hostshare 5.9T 8.7G 5.9T 1% /mnt/hostshare
rpool/hostshare 5.9T 8.7G 5.9T 1% /mnt/hostshare
Last edited:
