Initial installation: file system setup advise needed

[cosmos]

Hello,

back in the day I used to utilize LVM for storage. A couple of PVE hosts I've setup were nothing special regarding their hardware and articles regarding file system format seemed to advise using LVM for a variety of reasons.

Now I've got a one year old Proliant DL360 Gen 10 with two 240Gb SSDs in a hardware RAID mirror, with the controller being a HPE SmartArray P408i-a SR Gen10. The system also includes a 4Tb volume comprised from 3 disks in hardware RAID5, with the same RAID hardware controller.

I know that PVE has evolved a lot and now includes ZFS as an option. However I'm clueless as to when ZFS or another file system type is recommended. Any help provided will be appreciated.

 

[waltar]

It depends on what kind of storage you want to use, block (raw images) or file storage (qcow2 images).
If you want block storage you can take zfs with zvols (p408i in hba mode) or otherwise your hw-raid5 with lvm (without a filesystem).
If you want file storage again zfs (p408i in hba mode) or hw-raid5 with ext4, btrfs or xfs ontop. I prefere the last but everybody has it's own favorite and all variants are working.

 

[cosmos]

I'm gonna have a Windows Server 2022 as well as a Linux VM for starters (not much, I know). Would you prefer to set these up with block or with file storage?

And what about the PVE boot disk setup? ZFS-something or something else?

 

[waltar]

As said before I would take file storage in xfs as the file handling is so easy like cp, mv etc, take reflink copies as snapshots, cp them etc.
The "pve default" is to use block in lvm or since zfs available zvols but for me I don't like much the handling of blocks.
As boot setup I prefere a hw-raid1 with xfs also. If you want zfs mirror don't take not-enterprise ssd's as the pve db eat them.
Single disk to boot is possible but you got bunch of work to your environment if broken.

 

[cosmos]

I take then that you prefer qcow2 for VM storage? If so, any specific reasons for electing this path?

EDIT: There's this thread that does not encourage the use of this controller for ZFS: https://forum.proxmox.com/threads/d...rray-p408i-a-sr-gen10-in-hba-mode-zfs.136214/

If that's the case I'll go for whatever PVE recommends...

 

[waltar]

As we decided to use shared nfs storage qcow2 is the only option for vm's while lxc's still do raw also.
It's easy to handle this filesystem files, try somethink local, mv that to other host or further on to nfs, have reflink snapshots too, there's no problem with how get a machine out of a local disk of died pve node nor which vm/lxc uses all the space in lvm/lvm-thin, how to reduce that and all that hussle. It's all just normal files without the headache in any crash case while users ask when back ready.