RAID5 with LVM

[megagolgoth]

Hi,

Is it possible (or plan to make it possible) to install Proxmox with a RAID5 setup with LVM ?

LVM is supporting RAID5 with at some point on the configuration process "lvcreate -n grappe1 –type raid5 -L 10G -i 4 vg1".

ZFS can do RAID5-type, but I'm using consumer grade SSD, and this is not recommended (first performance issue, with big "IO delay"). The PC is an HP 400 G1, with i7 4790, 16Go of RAM, and 3 250GB SSD (Western Blue). It's for Yunohost/pfSense/HomeAssistant mainly. The energy efficiency, size and noise (WAF) are really important.

Bye !

 

[alexskysilk]

Is it possible (or plan to make it possible) to install Proxmox with a RAID5 setup with LVM ?

yes.

LVM is supporting RAID5 with at some point on the configuration process "lvcreate -n grappe1 –type raid5 -L 10G -i 4 vg1".

not quite. you need to use mdadm to make the underlying raid.

ZFS can do RAID5-type, but I'm using consumer grade SSD, and this is not recommended (first performance issue, with big "IO delay"). The PC is an HP 400 G1, with i7 4790, 16Go of RAM, and 3 250GB SSD (Western Blue). It's for Yunohost/pfSense/HomeAssistant mainly. The energy efficiency, size and noise (WAF) are really important.

dont use a single parity volumeset, ESPECIALLY with consumer grade drives. you're better off making a single mirror, and use the third drive for other purposes.

 

[megagolgoth]

mdadm, noted.

Software raid with three consumer grade SSD, isn't that better than only one consumer grade SSD ?

 

[LnxBil]

mdadm, noted.

Also note, that you cannot use software raid with mdadm in the PVE installer. You need to use the Debian installert and install PVE on top if you want to use it for the OS.

Software raid with three consumer grade SSD, isn't that better than only one consumer grade SSD ?

Sure, but we're talking about tree-vs-two, not anything-else-vs-one. Consumer SSDs in any enterprise environment are slow and will fail faster due to the write amplication, so if you're unlucky, your 3 identical SSDs will fail at the identical time.

 

[megagolgoth]

Yes, i've saw this. I need to install Debian, and after Proxmox on top of it.

It's for home use : Yunohost (blog, cloud), OPNsense/pfSense, HomeAssistant.

I think it could be an option to add on the installer : Software RAID (with disclaimer). At home, it's reasonable to experiment, but having enterprise grade SSD is really expensive. Furthermore, entreprise grade SSD (I think commonly SAS) are not easy to use on a consumer grade PC, who is only SATA.

My setup is Dell 7020, i7 4790, 16Go of ram. It's compact, silent : a good WAF

 

[LnxBil]

having enterprise grade SSD is really expensive. Furthermore, entreprise grade SSD (I think commonly SAS) are not easy to use on a consumer grade PC, who is only SATA.

There are also SATA enterprise SSDs and they are not as costly as the SAS counterpart.

I think it could be an option to add on the installer : Software RAID (with disclaimer)

There is a software raid, it is called ZFS

 

[esi_y]

not quite. you need to use mdadm to make the underlying raid.

This does not work in Debian? Because...

https://docs.redhat.com/en/document...egment-types_configuring-raid-logical-volumes

 

[esi_y]

There is a software raid, it is called ZFS

This is like saying that there's a replacement for Saab automobile, only it has wings and weapon pods.

 

[esi_y]

I think it could be an option to add on the installer : Software RAID (with disclaimer). At home, it's reasonable to experiment, but having enterprise grade SSD is really expensive. Furthermore, entreprise grade SSD (I think commonly SAS) are not easy to use on a consumer grade PC, who is only SATA.

They literally go out of their way not to support it, ever:
https://forum.proxmox.com/threads/mdraid-o_direct.156036/#post-713584

There literally no sensible selection for 2280 "enterprise" SSDs anyways to use in some consumer hardware.

 

[LnxBil]

This is like saying that there's a replacement for Saab automobile, only it has wings and weapon pods.

You mean something like the Saab 37 Viggen? ... yet it still can roll on a normal road.

 

[esi_y]

You mean something like the Saab 37 Viggen? ... yet it still can roll on a normal road.

Even 39 is about as old as ZFS now. Sometimes people only want the road vehicle, somehow it's easier to drive.

 

[megagolgoth]

ZFS isn't recommended with consumer grade SSD.

I experimented, and i've got high IO delay with them, as described on the documentation.

It's for an at home setup. The Dell R6x0 doesn't have a good WAF (Wife Acceptance Factor), it's costly, large and loud...

 

[esi_y]

ZFS isn't recommended with consumer grade SSD.

I experimented, and i've got high IO delay with them, as described on the documentation.

This typically holds true for the old pieces with no DRAM (no HMB) and very slow NAND. It would work just fine on Gen4 NVMe with high endurance. ZFS will have wose performance than other filesystems because of the "features" on any storage.

It's for an at home setup. The Dell R6x0 doesn't have a good WAF (Wife Acceptance Factor), it's costly, large and loud...

It's a vacuum cleaner.

 

[alexskysilk]

This does not work in Debian? Because...

https://docs.redhat.com/en/document...egment-types_configuring-raid-logical-volumes

It might, but its just shorthand to create mdraid in one command. same difference.

 

[guletz]

so if you're unlucky, your 3 identical SSDs will fail at the identical time.

Hi,

You can write 2-3 TB o data with non-zero value on a single SSD, this task can minimise the probability that all SSD to fail at the same time.

Good luck / Bafta !

 

[esi_y]

It might, but its just shorthand to create mdraid in one command. same difference.

So he does not have to "use mdadm" is what I was getting at...

 

[LnxBil]

You can write 2-3 TB o data with non-zero value on a single SSD, this task can minimise the probability that all SSD to fail at the same time.

That's a really good idea. Thank you

 

[esi_y]

That's a really good idea. Thank you

Have you actually seen 3 identical SSDs go at that much identical time, in real life? They fail close to each other, if anything, several TBs won't make much difference.

 

[LnxBil]

Have you actually seen 3 identical SSDs go at that much identical time, in real life?

I don't run that many consumer SSDs (that's what we're talking about), yet I had them fail in different desktop machines right about a month after the 2-year warranty expired, so I always caution with consumer drives.

 

[esi_y]

I don't run that many consumer SSDs (that's what we're talking about), yet I had them fail in different desktop machines right about a month after the 2-year warranty expired, so I always caution with consumer drives.

Yes, I just do not think they will go at the exact same time, especially 3. They sure fail and unpredictably (as an SSD would, compared to HDD), but it's either early fails or then it lasts some time (and not go out at the same TBW, really, it's more dispersed.)