ZFS: question about partitioning

[Alessandro 123]

I've seen here: https://pve.proxmox.com/wiki/ZFS_on_Linux#_bootloader that PVE add ad partition for grub in the ZFS unallocated space

Any advantage about this?
Why, in a brand new machine, I can see ZFS configured in partition 2 (sda2) and at the same time, sda1 and sda9 are still used by ZFS?

Is not clear this kind of partitioning, that would require additional steps when replacing a failed disk (ZFS is unable to automatically partition the new disk due to custom partitioning scheme)

 

[fabian]

you need to store the boot loader information somwhere - PVE uses GPT formatted disks and creates a "BIOS boot partition" for Grub (it's only 1M big, at the start of the disk). the other two partitions (number 2 and 9) are for ZFS - partition 2 is the actual ZFS partition, partition 9 is a bit of reserved space which allows you to replace a disk with a slightly smaller one of the same "marketing size".

when replacing a disk, you need to replicate the GPT, re-randomize the UUIDs, run grub-install on the replacement disk and then zpool replace using partition 2 instead of the full disk.

all of the above only applies to the rpool (and only to the first mirrored pair of disks when using a striped mirror / raid10)

 

[Alessandro 123]

Clear. The only drawback is that ZFS isn't able to automatically partition any replaced disk

 

[jim.bond.9862]

as fabian point out, all of the structuring you asked about only applies to the rpool, or OS drive.
ZFS does not do partitioning on data pools. the "part1", "part2", "part9" structure only used on the boot/OS drive(s)
hence why ZFS is unable to repartition the replacement drive, you do not work with the whole drive on this level, you work with a part of the drive, a partition.

think about it in other file systems terms.
let say you building out a new machine, if you use windows you use FAT FS or NTFS, is you use Linux you have a choice of EXT3/4 or XFS or BTRFS or ZFS.
when you run install of OS it does several things when preparing the drive for installation.
#1. creates a partition table (MBR or GPT) this is true for all OS setups. for boot drives we need a pre-defined structure that is recognized by all BIOS in use today. MBR is older and can be used on all HDD up-to 2TB in size. anything bigger and system needs to use special tricks to full the BIOS, making it think the drive is actually smaller so it can boot and thena OS is taking over the processing.
#2. creates a special partition or reserving a space, to store the boot files. called "/boot". usually this partition/space is hidden is is not seen most of the time.
#3. loads the appropriate boot files in that space and set a pointer in predetermined bit on the drive that any BIOS can see and read.

in Windows the drive info simply looks like "MainPartition + reserved space for system" you do not even see the "/boot" space at all.

in Linux using lsblk you will see all reserved space on the drive including "/boot " as partitions.

Again, all of this only matter for boot drives. data drives do not need this complexity and usually have only single partition defined.

however, with ZFS, even data drives still have a little reserved space left to accommodate the difference in drive manufacturing and advertisements as a 2TB drive from one mfg can be slightly smaller or bigger than a 2TB drive from another mfg.

and here we are back to main idea.
when you using ZFS you will always have Partition9 as this is a reserved space to account for disk size differences. it is also there for compatibility with Illumos systems. if you feel you don't needed you can remove it, ONLY ON DATA DRIVES, there is a how-to for that.
on OS/Boot drives you should keep the default layout as is.
partition1 changes based on disk usage.
on data drives that is the main and only partition spanning the whole disk sanse the space in partition 9
on OS/boot drives partition 1 is used for /boot space. and the main partition is partition 2.

so in rpool, you use partition 2. hence why ZFS can not prepare the replacement drive for you. just like with any other raid technology, boot disk is a special case and needs to be handled differently.

 

[fabian]

the first partition is not where /boot is stored (just the first part of Grub is stored there), /boot is just a directory on the / file system on recent (IIRC, anything >= 4.x at least?) PVE installations. Grub can boot from LVM and ZFS nowadays, so a separate /boot is no longer necessary.

 

[Alessandro 123]

But if I don't need any MBR failover and boot only via UEFI, isn't possible to use standard ZFS partitioning?

ZFS automatically creates partition 1 (data) and partition 9 when using the whole disk. It seems possible to directly boot in zfs without any bios partition

 

[fabian]

the PVE installer does not support ZFS + UEFI, because there is no sane way to keep the ESP in sync (grub does handle multiple BIOS boot partitions just fine). so if you want to setup ZFS + UEFI, you need to do so yourself (and then you can use the full disks for ZFS and setup a separate single-point-of-failure ESP device where you install grub).