Does forcing 4Kn on VM disks actually help?

[mercifultape]

Running PVE on hosts with 512/512 disks, storage is ZFS (ashift=12) and Ceph.
Guests use virtio-scsi-single, mostly Linux, some Windows.

I wonder if it's worth adding (or similiar):

args: -global scsi-hd.physical_block_size=4096 -global scsi-hd.logical_block_size=4096

For Linux it feels pointless - page cache is 4K so guests should already send 4K
writes regardless - except maybe for O_DIRECT workloads, but most DBs should use >= 4K pages anyway.

Questions:
1. On 512/512 disks with ZFS (ashift=12) / Ceph underneath, does this give any performance gain for Linux/Windows guests?
2. Does it change anything on native 4Kn disks?
3. Has anybody benchmarked it and can provide results?

Thanks.

 

[psalkiewicz]

Hi, In terms of performance I wouldn't expect any measurable gain.

The real-world gain for Linux is mostly at partition alignment and filesystem mkfs time — tools like parted, fdisk, and mkfs.xfs/mkfs.ext4 read the reported physical block size and align accordingly. If your logical block size is 512, some tools may create slightly suboptimal layouts, though most modern tools default to 1MB alignment anyway, making this moot in practice.

It's good for the sake of correctness but I wouldn't expect any significant improvement. It's more of an academic discussion than real world tweak

 

[stanleybaxter]

Hi, In terms of performance I wouldn't expect any measurable gain.

The real-world gain for Linux is mostly at partition alignment and filesystem mkfs time — tools like parted, fdisk, and mkfs.xfs/mkfs.ext4 read the reported physical block size and align accordingly. If your logical block size is 512, some tools may create slightly suboptimal layouts, though most modern tools default to 1MB alignment anyway, making this moot in practice.

It's good for the sake of correctness but I wouldn't expect any significant speed stars improvement. It's more of an academic discussion than real world tweak

That makes sense, thanks for the clear explanation. Sounds like it’s more about doing things “right” rather than expecting real performance gains, especially with modern tools already handling alignment well. Good to know it’s not something worth over-optimizing in most setups.

 

[leesteken]

I use -global scsi-hd.physical_block_size=16k -global scsi-hd.logical_block_size=4k to match the volblocksize size and vdev size (respectively). GRUB (in Linux VMs) sometimes complains about sector size 4096 not supported and some older Linux distro's did not install properly because they assumed a 512 sector size. It works (and sounds efficient to me) but I don't have performance figures.

 

[mercifultape]

Hello,

I've done some testing. SSDs reporting 512-byte sectors using ceph. VM1 was installed without any qemu args, VM2 was installed after adding "args: -global virtio-blk-device.physical_block_size=4096 -global virtio-blk-device.logical_block_size=4096 -global scsi-hd.physical_block_size=4096 -global scsi-hd.logical_block_size=4096" into VM2's qemu config. Confirmed the physical/logical sector size in VMs via lsblk.

4k random direct writes performance was basically the same for 512 and 4kn throughout multiple runs with difference about 1% - which might be statistical error. Other tests - random read, sequential read/write, fsync showed even smaller differences.

I might test that on different PVE cluster using ZFS next time.

Thanks everybody for your inputs.