ZFS on top enterprise hardware RAID is bad: any evidence or first-hand experience?

[Tony]

We see everywhere the advice "don't use ZFS on top hardware RAID". But I also see opinions that it's a myth.

This question is not about the rationals to avoid (or use) hw raid with ZFS. I'd like to hear from Proxmox community about first-hand experience with ZFS on top hw raid, especially enterprise hw raid.

I have been running a Proxmox cluster of a few Dell PowerEdge servers, with ZFS on top hw raid10 (Dell PERC). I have not experienced any problem (till now), but I keep asking myself whether ZFS on top HBA brings any advantage. The most common argument is that ZFS can detect bit rot in HBA mode and hence offers better data protection. But it seems hw raid cards do that, too. Performance-wise is enterprise hw raid better.


Links:
ZFS is Perfectly Safe on Hardware RAID | MangoLassi
Bit-rot and RAID - Alastair’s Place (alastairs-place.net)

 

[Klaus Steinberger]

It is not a myth, just don't use a Raid Controller or anything similar below a ZFS Pool. ZFS likes to see all disks directly. Some features even require this, for example the protection against silent bitrot.

Just use adequat SAS oder NVME HBA's

 

[Dunuin]

It is not a myth

There is a dediated section in the ZFS documentation explaining this: https://openzfs.github.io/openzfs-d...uning/Hardware.html#hardware-raid-controllers