Oh, ok... I knew that truly 4K native spinning disks are few, but I'm probably not up to date because I haven't used spinning for a long time.
Yet another "ZFS on HW-RAID" Thread (with benchmarks)
- Thread starter EdoFede
- Start date
-
- Tags
- hardware raid performance zfs
Oh, ok... I knew that truly 4K native spinning disks are few, but I'm probably not up to date because I haven't used spinning for a long time.
> So personally I see two possibilities:
https://www.reddit.com/r/truenas/comments/zz3sfg/hardware_raid_zfs_shoot_me_i_deserve_it/
https://www.truenas.com/community/r...bas-and-why-cant-i-use-a-raid-controller.139/
https://www.reddit.com/r/zfs/comments/135ku0i/zfs_on_top_of_multiple_hwraid0s/
https://openzfs.github.io/openzfs-d...uning/Hardware.html#hardware-raid-controllers
> On another three servers with ZFS on a proper HW-Raid controller, I've got already 2 disk failure without any issue.
Replaced, rebuilded, not a single error.
Every few months, some subgenius drops in and thinks they know everything. No, you've actually built a house of cards on quicksand. The wind just hasn't come by yet. You're going to lose your data when it falls over and sinks - and you won't be able to migrate your disks to anything but the same make and model of RAID card. Please do not encourage others to try replicating your foolishness.
- ZFS is so unreliable that it cannot work properly on hardware solutions (on which every other file systems works fine)
- The recommendation is simply derived from an exaggerated interpretation of what is actually a very interesting feature, but not so critical in its absence
https://www.reddit.com/r/truenas/comments/zz3sfg/hardware_raid_zfs_shoot_me_i_deserve_it/
https://www.truenas.com/community/r...bas-and-why-cant-i-use-a-raid-controller.139/
https://www.reddit.com/r/zfs/comments/135ku0i/zfs_on_top_of_multiple_hwraid0s/
https://openzfs.github.io/openzfs-d...uning/Hardware.html#hardware-raid-controllers
> On another three servers with ZFS on a proper HW-Raid controller, I've got already 2 disk failure without any issue.
Replaced, rebuilded, not a single error.
Every few months, some subgenius drops in and thinks they know everything. No, you've actually built a house of cards on quicksand. The wind just hasn't come by yet. You're going to lose your data when it falls over and sinks - and you won't be able to migrate your disks to anything but the same make and model of RAID card. Please do not encourage others to try replicating your foolishness.
Congratulations, great behavior for a technical discussion forum!
You simply didn't even read what I wrote in the entire thread, not even the very clear bolded parts.
If you are not able to deal with a technical dialogue by discussing topics without offending, your contribution is totally useless.
So Summary, the scenarios:
1., ZFS on RAW disks
2., ZFS on HWRAID
3., ZFS+SLOG on RAW disks
4., ZFS on OpenCAS (RAW disks)
I would be interesting to see the 3., and 4., scenario benchmarks.
The idea of the OpenCAS is the same for SLOG, but more robust ( middle-ware between the raw disk and zfs, basically a "smart software cache" solution)
The difference between 1., and 4., :
[ disk ] ----> [ zfs+slog ] -----> [ filesystem ] -----> [ VM ]
[ disk ] ----> [ opencas ] ----> [ zfs ] -----> [ filesystem ] -----> [ VM ]
The OpenCAS scenario, the separate "cache" device can be a "raid" (software/hardware) disk too (example: 2x NVME in raid-1 ) - while acting a proxy between the ZFS and the RAW disks.
1., ZFS on RAW disks
2., ZFS on HWRAID
3., ZFS+SLOG on RAW disks
4., ZFS on OpenCAS (RAW disks)
I would be interesting to see the 3., and 4., scenario benchmarks.
The idea of the OpenCAS is the same for SLOG, but more robust ( middle-ware between the raw disk and zfs, basically a "smart software cache" solution)
The difference between 1., and 4., :
[ disk ] ----> [ zfs+slog ] -----> [ filesystem ] -----> [ VM ]
[ disk ] ----> [ opencas ] ----> [ zfs ] -----> [ filesystem ] -----> [ VM ]
The OpenCAS scenario, the separate "cache" device can be a "raid" (software/hardware) disk too (example: 2x NVME in raid-1 ) - while acting a proxy between the ZFS and the RAW disks.
Last edited:
>To be clear, just last week I installed two servers with ZFS, flashing the PERC H330 controllers in IT mode (HBA330), because of the absence >of battery backed-up cache.
No need "IT mode", just use "WriteThrough" mode if battery backed-up cache absent (not writeback).
No need "IT mode", just use "WriteThrough" mode if battery backed-up cache absent (not writeback).
Last edited:
Bullshit. Example : newer lsi raid-card is backward-compatible with older lsi raid-cards. I myself replaced old dead raid-card some years ago. After reboot it prompts to "import foreign configuration" (which is stored on hw-raid10 virtual drive) and all ok.
This thread started as performance comprising between native ZFS raid and ZFS on top of H.Raid and half leg is it good to run ZFS on H.Raid.
I can tell you this - How you will set up that's way it will work.
If you want H.Raid to monitor HDD, I can suggest to set ZFS checksum=off otherwise ZFS will return IO error to software on checksum mismatch because ZFS will not have how to fix it.
One of my personal setup from 'a horror story' book.
As you can see stuffs can work in different ways
I can tell you this - How you will set up that's way it will work.
If you want H.Raid to monitor HDD, I can suggest to set ZFS checksum=off otherwise ZFS will return IO error to software on checksum mismatch because ZFS will not have how to fix it.
One of my personal setup from 'a horror story' book.
In one VM (external) I have limit in HDD space. I tried other things but I ended with creating ZFS ( in VM ) from files and files are accessed through SAMBA. Samba server is not local too
Files in samba server -> VM ZFS import pools
This way system is working for about 3 years. I`m planning to replace it. But as for this journey I had couple times to restore from backup due checksum problems. I do snapshot every 10 minutes to minimize the gap.
As you can see stuffs can work in different ways
What a strange recommendation!
I want my data delivered to me to be intact. If it got damaged by any means I want at least to know this fact - and not hidden!
Of course: your mileage may vary. Perhaps there may be circumstances where this may help reading "half destroyed" data. But for nomal usage on a daily basis this sound really like a bad advise.
Just my 2 €¢...
If you want to know then yes, keep it on. In the same situation others FS will continue to operate.
But keep in mind your VM will be broken or you movie will be half way watchable.