Recommended ZFS pool setup?

[Toxik]

Hi,

I want to use an HPE Microserver for Proxmox. It can take 5 drives. I want Proxmox and my VMs on a ZFS pool with redundancy. In my understanding I could create a rpool like the following and use mirror-0 for Proxmox and raidz1-0 for the VMs:

rpool
  mirror-0
    ssd1-256GB
    ssd2-256GB
  raidz1-0
    ssd3-2TB
    ssd4-2TB
    ssd5-2TB

-> 256 GB for Proxmox, 4 TB for VMs

An alternative would be the following and just install Proxmox and the VMs on the same raidz2-0:

rpool
  raidz2-0
    ssd1-1TB
    ssd2-1TB
    ssd3-1TB
    ssd4-1TB
    ssd5-1TB

3 TB for everything, but probably higher redundancy.

So my questions:
1. Did I understand this right?
2. If yes, what is the recommended way?
3. Or is there another, better alternative?

EDIT: Second example was meant to be raidz2, not raidz1.

 

[wahmed]

My personal recommendation is Mirror for Proxmox OS drives and Z1 for the VMs. This keeps the OS drives separate from everything. Much easier to work with. I personally do not prefer Z1 at all since it only gives 1 drive failure redundancy. But it is understandable in your case due to lack of extra bays.

Also keeping the Z1 separate from the OS drive you have the freedom to simply move the array of Z1 on another machine if you need to in future to increase capacity.

 

[Nemesiz]

rpool
  mirror-0
    ssd1-256GB
    ssd2-256GB
  raidz1-0
    ssd3-2TB
    ssd4-2TB
    ssd5-2TB

-> 256 GB for Proxmox, 4 TB for VMs

This is single pool and its very unbalanced in every way.


You mush choose what you want. If your bugged allows 5 x big SSD then do Raidz2.

 

[Toxik]

This is single pool and its very unbalanced in every way.

Oh, I guess what I actually mean is this:

rpool
  mirror-0
    ssd1-256GB
    ssd2-256GB

vmpool
  raidz1-0
    ssd3-2TB
    ssd4-2TB
    ssd5-2TB

rpool -> Proxmox rootfs
vmpool -> VM storage

 

[Toxik]

It seems from a performance point of view one should have everything (Proxmox + VMs) on a single rpool, because then higher IOPS could be reached. Pool access is distributed across more disks then.
Just let ZFS do it's magic and don't bother about seperated file systems. Seperate datasets are fine.

Any other opinions?

 

[Nemesiz]

Yes, more disk = more speed/IO. But creating ZFS you must think about how much time you have until you change death/damaged disk. Like Raidz3 give more time compare to others options. I had strange situation then one disk go offline because of unknown reason and ZFS stops writing to it. Pool become DEGRADED until I do zpool clear to make the pool do resilvering and back to good state.

 

[guletz]

Hi,

Because you intend to use SSD only, thake in account that in case of failed ssd, the resilver time will be very short compare with rotational hdd.

Yes, more disk = more speed/IO

Is not that simple. Using raidzX with 3 vdev or 5 vdev, the IOPs it will be the same. Maybe for your own case (we do not know if you want to use VM or CT or mixed), it will be better to have 2 different pools (mirror with 2 ssd for root and vm/ct that need higher iops and a raidz1 for vm/ct where lower iops is ok)

Good luck