Recommended ZFS pool setup?

Toxik

Well-Known Member
Jul 11, 2019
56
7
48
Germany
Hi,

I want to use an HPE Microserver for Proxmox. It can take 5 drives. I want Proxmox and my VMs on a ZFS pool with redundancy. In my understanding I could create a rpool like the following and use mirror-0 for Proxmox and raidz1-0 for the VMs:
Code:
rpool
  mirror-0
    ssd1-256GB
    ssd2-256GB
  raidz1-0
    ssd3-2TB
    ssd4-2TB
    ssd5-2TB
-> 256 GB for Proxmox, 4 TB for VMs

An alternative would be the following and just install Proxmox and the VMs on the same raidz2-0:
Code:
rpool
  raidz2-0
    ssd1-1TB
    ssd2-1TB
    ssd3-1TB
    ssd4-1TB
    ssd5-1TB
3 TB for everything, but probably higher redundancy.

So my questions:
1. Did I understand this right?
2. If yes, what is the recommended way?
3. Or is there another, better alternative?

EDIT: Second example was meant to be raidz2, not raidz1.
 
Last edited:
My personal recommendation is Mirror for Proxmox OS drives and Z1 for the VMs. This keeps the OS drives separate from everything. Much easier to work with. I personally do not prefer Z1 at all since it only gives 1 drive failure redundancy. But it is understandable in your case due to lack of extra bays.

Also keeping the Z1 separate from the OS drive you have the freedom to simply move the array of Z1 on another machine if you need to in future to increase capacity.
 
Code:
rpool
  mirror-0
    ssd1-256GB
    ssd2-256GB
  raidz1-0
    ssd3-2TB
    ssd4-2TB
    ssd5-2TB
-> 256 GB for Proxmox, 4 TB for VMs

This is single pool and its very unbalanced in every way.


You mush choose what you want. If your bugged allows 5 x big SSD then do Raidz2.
 
It seems from a performance point of view one should have everything (Proxmox + VMs) on a single rpool, because then higher IOPS could be reached. Pool access is distributed across more disks then.
Just let ZFS do it's magic and don't bother about seperated file systems. Seperate datasets are fine.

Any other opinions?
 
Yes, more disk = more speed/IO. But creating ZFS you must think about how much time you have until you change death/damaged disk. Like Raidz3 give more time compare to others options. I had strange situation then one disk go offline because of unknown reason and ZFS stops writing to it. Pool become DEGRADED until I do zpool clear to make the pool do resilvering and back to good state.
 
Hi,

Because you intend to use SSD only, thake in account that in case of failed ssd, the resilver time will be very short compare with rotational hdd.

Yes, more disk = more speed/IO

Is not that simple. Using raidzX with 3 vdev or 5 vdev, the IOPs it will be the same. Maybe for your own case (we do not know if you want to use VM or CT or mixed), it will be better to have 2 different pools (mirror with 2 ssd for root and vm/ct that need higher iops and a raidz1 for vm/ct where lower iops is ok)

Good luck
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!