Zfs and Hardware Raid.

fusion

Member
Nov 4, 2015
36
0
6
Spain
Hi, I'm renting some servers with hardware raid.

RAID controller (P222)
RAID controller (P420)

Configured as hardware RAID 0.

I'm using ZFS (RAID1), what danger can I have ?, or will I only lose performance, a few days ago I use it like this and at the moment everything normal.

Thanks for your advice.

regards
 
Hi,

it is hard to say what happens, it dependence on the HWRaid Firmware.

But general it is not recommended from anyone.
See ZFS user list.
 
Bad experience with ZFS and Hardware Raid ..
Errors on FS can cause the detach of ZFS raid, but the controller can't sign the disk as failed
 
ZFS and hardware raid means data loss for sure.
ZFS must know exactly when data are wrote on each disks and this won't be possible with hardware with batteries.

An hardware raid with no battery doesn't give you more performance than a ZFS RAID but still hide some low-level command to ZFS

No, you must not use ZFS on any kind of hardware. ZFS *must* manage disks on it's own all levels.
 
Additionally, with hardware raid you are forced to always use that kind of controller. If you have to change the server in a rush, with ZFS you only have to move disks from server1 to server2, with hardware raid you also have to use the same controller with the same firmware (or a newer one) and there is also a risk for some incompatibilites
 
Although I have read/seen its not recommended I have considerable experience with running zfs ontop of hardware raid (LSI Cards). We have over a 500TB of hardware raid 5 storage which zfs is sitting on top of. We have been running this configuration now for almost 5 years with absolutely no issues. We have replaced drives/enclosures and never had a single issue. The only reason we went this route was to take advantage of zfs compression.

In fact we are presenting the raid 5 storage over 10G iscsi, then putting zfs on top. Im sure people will hate, but the proof is in the pudding, it just works and with the amount of storage/activity we have going on, if it was going to cause an issue, we would have seen it by now. Maybe we are just lucky, but thats alot of storage to be lucky on.
 
500TB in a single raid5 ? You are playing with the fire, really.
RAID-5 limit is about 12TB, because current disks has URE about 1^14 that is about 12TB.

Resyncing more than 12TB means to hit an URE in another disk almost for sure.

Yes, you have been lucky
 
500TB in a single raid5 ? You are playing with the fire, really.
RAID-5 limit is about 12TB, because current disks has URE about 1^14 that is about 12TB.

Resyncing more than 12TB means to hit an URE in another disk almost for sure.

Yes, you have been lucky

No multiple raid 5 setups, multiple enclosures. That is just the total space we have with zfs sitting on top of hardware raid. It just works, has nothing to do with luck. If we were getting lucky, we would have been burned years ago.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!