ZFS recommended config for migrating to 4 NVMe disks

[luison]

Hi, currently migrating a local testing server from PVE7 (LVM) to PVE8 ZFS.

We have:

• a new 2TB NVMe disk (blank)
• 3x1Tb NVMe disks (data including hybrid raid on one partition and PVE vgroup)
• 1x1Tb SSD disk (data, same as above)
• Additional HDDs for backup

My objective is to create one or two big pools for all data, perhaps 3xNVM for system and thin partitions and core data and another for the rest.

I am uncertain which would be the best final RAID ZFS format to end up with, considering to that ideally I need to migrate the data step by step. This is, perhaps, start with the new and one of the older NVMe disks, migrate data, and then add the other two to the pools. I also have the issue of currently having root under pve vgroup, which means its shared with my thin lvm so I would need to reduce it.

Without ZFS I would go for a RAID 1+0, so I could build the system with the two available disks and then add their mirrors. I am guessing I can do the same with ZFS but not that sure of the process.

So the questions are:
- what would be the ideal ZFS raid format for a main pool having 4 x 900Gb NVMe disks + one 900Gb SSD disk (I would still have anothe 1Tb NVMe spare)
- alternatively, what would it be for three, so I could consider one 900Gb x 3 disks pool plus another one (sharing the first disk) of 2xNVME + 1 SSD)
- in either case can I create the required pools and then add the spare/additional ones, once they are cleared of data?

Thanks.

 

[v95klima]

I would take two of the 1TB NvME and create a zfs mirror for strictly back ups.
The rest I would keep all as single disk, one modern fast NvME for Proxmox on ext4 and the rest as zfs stripe working disk.