Best ZFS configuration for 4 SSDs

[marcelovvm]

Dear colleagues, I'm setting up my first server with Proxmox in a test environment to later go to production. I've already used Proxmox at home in a simpler environment (only one XFS disk for the system and for storing the VMs). But this new scenario is very different and I want to try ZFS.

The server has a flash drive (32GB) with Proxmox 8 installed. I have 4 Kingston SDEC450R SSDs (natively 512 bytes per sector) of 480GB connected to an LSI 9211-8i controller. When we used RAID this server was configured in RAID 10 having 899GB RAW for the Linux and Windows VMs (we used ESXi). The server has 72GB of RAM.

So for ZFS what would be the best configuration for good performance? Two pools + mirror? A RAID-Z1 or Z2? For the ashift parameter, would 12/13 be better?

Conditions:
- losing the space of two disks (960GB) is acceptable;
- for protection, losing a disk and keeping the environment working is quite acceptable (even knowing that in RAID 10 you could lose up to two disks)

Live long and prosper,
Marcelo Magalhães

 

[ness1602]

Usually it is really(REALLY) not recommended to run proxmox on usb drives.Enough on that.
For ZFS raid10 would be recommended way, for performance,and ideally you can lose two nodes(not in all cases).

 

[marcelovvm]

Usually it is really(REALLY) not recommended to run proxmox on usb drives.Enough on that.
For ZFS raid10 would be recommended way, for performance,and ideally you can lose two nodes(not in all cases).

OK. So just in case you opt for a RAID 10-like structure. Would I have to create two VDEVs with two disks each and a mirror between those two VDEVs? Stripe + mirror.

 

[ness1602]

YEs,create classic Raid10.

 

[Neobin]

OK. So just in case you opt for a RAID 10-like structure. Would I have to create two VDEVs with two disks each and a mirror between those two VDEVs? Stripe + mirror.

What you describe is a raid01.

A raid10 mirrors the disks per VDEV and then stripes the VDEVs together.

[0] https://www.45drives.com/community/articles/RAID-and-RAIDZ

 

[marcelovvm]

What you describe is a raid01.

A raid10 mirrors the disks per VDEV and then stripes the VDEVs together.

[0] https://www.45drives.com/community/articles/RAID-and-RAIDZ

Good point! Tks.

So... the sequence of commands would be:

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sdc /dev/sdd

 

[Dunuin]

Good point! Tks.

So... the sequence of commands would be:

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sdc /dev/sdd

In ZFS a raid10 i called a "striped mirror", not a mirrored stripe

And no, that would be wrong and would try to create two mirrors (aka raid1). Striped mirror (aka raid10) would be zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb mirror /dev/sdc /dev/sdd. And don't use "/dev/sda", "/dev/db" and so on. Use "/dev/disk/by-id/YourFirstDisk", "/dev/disk/by-id/YourSecondDisk" and so on instead.

 

[marcelovvm]

In ZFS a raid10 i called a "striped mirror", not a mirrored stripe

And no, that would be wrong and would try to create two mirrors (aka raid1). Striped mirror (aka raid10) would be zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb mirror /dev/sdc /dev/sdd. And don't use "/dev/sda", "/dev/db" and so on. Use "/dev/disk/by-id/YourFirstDisk", "/dev/disk/by-id/YourSecondDisk" and so on instead.

TKS!!! Thank you for your help! I hadn't noticed the "subtlety" of the difference.