NVME Sector Size - Can I change this once the disk is already in a ZPool RAID 10

[jimbob]

I have installed proxmox using the GUI Installer and missed what I believe to be an important setting for the sector size of the nvme disks.

When running

smartctl -a /dev/nvme1n1

it shows a + next to 512, and the open-zfs docs recommend the disks should be formatted with a sector size of 4096 byte sectors in order to be more performant and reduce ware.

As each disk is 3.84TB, am I right in thinking this will make a difference.

As I have 4 x NVME in a RAID 10, would it be worthwhile evicting the disk, formatting it with a 4k sector size and re-presenting the disk back to the zpool. I don't have much data on the disks yet, but do have the boot and root partitions on there so not as straight forward as destroying the pool and re-creating it.

Or is there a way to change on the fly?

 

[Dunuin]

With ZFS ashift is defining what the smallest blocksize is ZFS will use. zpool get ashift rpool should show you what blocksize ZFS is using for the disks. "ashift=12" would be 4K and "ashift=9" would be 512B. The ashift can't be changed without destroying and recreating the whole pool.
And then you sometime got the option to tell the disks firmware to optimize or 512B for 4K blocks (internally then will still use a blocksize that is way higher than 4K...thats why you should get enterprise SSDs that can cache sync writes in RAM because of the powerloss protection, so you don't got a horrifying write amplification when doing small sync writes). Never had to this, but I would guess changing this will also wipe the complete disk, like a secure erase command would do to start new with a fresh factory reset.

 

[koalillo]

As this shows high on Google... I was following https://openzfs.github.io/openzfs-docs/Performance and Tuning/Hardware.html#nvme-low-level-formatting , and decided to reformat as 4k... then the Proxmox installer said that booting off 4k-formatted NVMe with ZFS was not supported. Undoing that now