RAID0 ZFS over hardware RAID

[Dan Nicolae]

Hello, everyone.

I have to setup an Proxmox VE 6.2 cluster that use local disks as storage for VM's (KVM). The local storage is an hardware raid array (8 HDD in RAID0 on a Dell PERC H70 mini). We would like to use the live migration feature with this local storage that is available on ZFS only. Can I setup safely ZFS RAID0 with only one disk (the HW RAID array)? Would i work without any issues? I search the web and there are a lot of discussions on this.

Thanks for the answer.

 

[aaron]

Do I understand you correctly? The PERC is configured to have 8 disk in one RAID0 (striping)?

 

[Dan Nicolae]

Looks like. It's not my creation. I'm just the software guy.

 

[Dan Nicolae]

I'll ask them to reconfigure the array, I know that is just crazzy. But the question remains, is ok to use RAID0 ZFS on a hardware RAID?

 

[aaron]

If that really is the case, then there is not safety at all! Once a single disk fails, the whole RAID is gone.\

But the question remains, is ok to use RAID0 ZFS on a hardware RAID?

No.

The general rule is, to give the disks to ZFS (and Ceph as well) as naked as possible. Ideally that is through a simple SATA controller or an HBA flashed to IT mode.

If the RAID controller can be set to HBA mode or pass through the disks raw you will gain a lot more. Disable all caches on the RAID controller though.

The one workaround that people try to do, especially on older HP RAID controllers which don't have a simple way to pass through disks directly or don't supporting booting from RAID controllers set to HBA mode is to create a single RAID0 for each disk. This collection of RAID0 will then in turn be used to create the ZPOOL.

Even this setup is problematic because the RAID controller abstracts the disks and usually has its own caches. These caches can cause data inconsistencies and lower performance if they work against ZFS. Not to mention the much more complicated process to replace a failed disk.

If you set up ZFS on top of a RAID consisting of multiple disks, you prevent it to use the self healing mechanisms of ZFS and performance is usually lower as well.

With all that said, we will not stop anyone wanting to still do it. Sooner or later though there will be problems.

 

[Dan Nicolae]

Thanks for the answer.

 

[guletz]

Salut Dane,

If and only if your controller have a BBU (baterie) and you have a decent UPS, you could create a raid0 (from your raid controller) , for each of your hdd. Then you could create a zpool using all this raid0 drive. If the raid controller is a decent one, more likely you do not have problems in the future.

But to be on safe side you need to have backups.

Bafta / Good luck!

 

[Dan Nicolae]

Always is a good ideea to have backups.
Multumesc/ Thank you.