Should i install proxmox on my 4TB SSD RAID 5 setup ?

[moda20]

I am moving my homelab from docker containers to proxmox VE VMs. my motivation is to be able to backup whole folders / images more easily and not have 5 or 6 different postgres containers at the same time. A secondary goal would be to learn mopre about virtualization (Okey knowledge).

My goal is to have a stable drive to use. i had prior issues with bad disks failing and breaking my filesystem. i am going to get 3 2TB SSDs and put them in RAID 5 with 4Tb of usable space.

I want to know Hoiw can i install it on the RAID setup and if the RAID config is created by the installer or should i have it ready somehow.
the second question is shoudl i do the isntall on the full drive ? knowing that i don't have any other SSDs and my HDDs are nor fast enough to be used for other than bulk storage and maybe backups and temporary downloaded files.

Thanks in advance.

 

[d.oishi]

If you are using a real hardware RAID controller and you want to use hardware RAID, then the RAID5 array should be configured in the RAID controller firmware first. During the Proxmox installation, that array should appear as a single logical disk.

In that case, I would not put ZFS on top of the hardware RAID array. Just install Proxmox on the logical disk using the normal non-ZFS installer option, for example ext4/xfs with LVM-thin.

If you are not using a hardware RAID controller, or if your controller can expose the disks individually, for example in JBOD/HBA/IT mode, then you can let the Proxmox installer create a ZFS RAIDZ-1 pool using the three SSDs.

Since you only have three SSDs, it is fine to use all three of them for the Proxmox installation and VM storage in the same ZFS pool. The installer will handle the system and VM storage layout on that pool.

However, RAIDZ-1 can tolerate one disk failure.

This manual section may also help:
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#installation_installer

 

[leesteken]

RAIDz1 is terrible for VMs and drives without PLP are also no fun with ZFS. Hardware RAID5 (with battery backup for sync write caching) without ZFS is fine.
https://forum.proxmox.com/threads/fabu-can-i-use-zfs-raidz-for-my-vms.159923/ (fixed, thanks to @Johannes S)

 

[Johannes S]

RAIDz1 is terrible for VMs and drives without PLP are also no fun with ZFS. Hardware RAID5 (with battery backup for sync write caching) without ZFS is fine.
https://forum.proxmox.com/threads/fabu-can-i-use-zfs-raidz-for-my-vms.159923/

Working Link: https://forum.proxmox.com/threads/fabu-can-i-use-zfs-raidz-for-my-vms.159923/

 

[alexskysilk]

RAIDz1 is terrible for VMs and drives without PLP are also no fun with ZFS.

words like "terrible" are terrible. they provide no context. You could say "RAIDz1 performs poorly compared to RAID10" or "RAIDz1 provides poor risk management vs RAIDz2" but without knowing the use case and expected performance and fault tolerance requirements any absolute judgment fails.

With regards to plp- zfs handles power loss as well as any other file system. you might just lost a second of pending writes. Do you want it? yes. do you need it? same question as above...

Hardware RAID5 (with battery backup for sync write caching) without ZFS is fine.

The biggest issue with RAIDz1 is single parity; RAID5 buys you nothing. PLP obviates the need for a BBU.

 

[UdoB]

With regards to plp- zfs handles power loss as well as any other file system.

Yes.

you might just lost a second of pending writes.

Up to more than five seconds - that's the default interval for writing out (async-) buffered content. More than five because there are two Transaction Groups "TXG": one is active and receives data to be written (from a program) while the other is being emptied by writing its data to the final destination.

But this (power loss) is only one aspect of having PLP. The other one has the same importance for me: it makes sync writes multiple times faster. The reasoning has been discussed multiple times. Cheap SSD w/o PLP under load may feel slower than rotating rust. Yes. Been there, felt that ;-)

The third aspect is durability: with PLP the same SSD would work much, much longer as the actual writes of the internal groups-of-cells happens less often and can get sorted in a more clever way. Remember that these groups alway needs to get read(!) completely, some content will get modified and then the whole shebang has to be written back to where it belongs. That's slow by definition..., and the Copy-on-Write approach of ZFS does not help too...

 

[leesteken]

words like "terrible" are terrible. they provide no context.

The problem with RAIDz1 is the lack of (read) IOPS compared to mirrors while putting multiple machines worth of IO to it (and often less than expected usable space). I added a link with more detailed discussing and the "terrible" was just my (opinionated) summary. Unfortunately, the link was broken but this has been fixed.

 

[alexskysilk]

Up to more than five seconds

true.

it makes sync writes multiple times faster.

If you want to buy an 18 wheeler for haul groceries thats your call- it doesnt mean you need an 18 wheeler. match the solution to the problem. Not every usecase benefits.

The third aspect is durability: with PLP the same SSD would work much, much longer as the actual writes of the internal groups-of-cells happens less often and can get sorted in a more clever way

Thats an argument for a newer drive, not for a plp. technology doesnt stand still.

The problem with RAIDz1 is the lack of (read) IOPS compared to mirrors while putting multiple machines worth of IO to it (and often less than expected usable space).

See that makes a lot more sense