So yesterday i was deciding whether i was gonna use HW or SW RAID. I've figured out i'd like to use the SW RAIDZ-2 With 2 parities and i'd like to have 1 parity as hotspare. Does proxmox support hotspare and how can i configure it with RAIDZ-2? Any advice is greatly appreciated.
Hotspare configuration
- Thread starter BryanVeluwe
- Start date
-
- Tags
- hotspare raidz2 software raid
In this case I would recommend RaidZ3! Your planned-as-"hot spare"-disk is being integrated continuously, so no re-silvering necessary; three disk may fail without data loss.
This is a decision that can only be made when creating a new vdev. Changing the mode from RaidZ1/Z2 to Z3 is not possible.
Edit: you didn't tell us your use case. Make sure to understand the implications regarding IOPS when deciding between multiple mirrors (aka Raid10) and RaidZx. Just saying...
Last edited:
Right, Let me elaborate the situation better. I was going into Proxmox with the idea of trying to use RAID-5 with 4 disks. Im very new to Proxmox and i don't have alot of knowledge about ZFS nor RAID. Since RAID-5 is not directly supported inside of Proxmox I would have to run it via the hardware RAID-controller. When i found out a lot of people value RAIDZ-1-3 (ZFS) above RAID 5 I changed my plans and went for RAIDZ-2 after researching what exactly RAIDZ is. My plan is to run Proxmox with 2 disks reserved for storage , 2 disks reserved for parity and i'll make 1 of the parity disks a hotspare incase that the other parity disk fails. I'm using a PowerEdge R240 from Dell.
Last edited:
Relevant is the type of the Raid card. For using ZFS you really want to flash it to "IT-Mode" - if possible. If the controller supports "pass-through-mode" this may be sufficient. The point is that ZFS wants to see the real disks, not some virtual representation, purely invented by the controller.
If you have to use the Raid-Controller as... well..., a Raid-Controller then you should go for LVM on top of it.
Personally I do whatever I can to use ZFS wherever possible. Additionally to what the Hardware-Raid can do by establishing "Raid5" you get several goodies like transparent compression, better snapshots, reliable bit-rot detection and repair and ZFS guarantees to deliver the same data tomorrow which you wrote to disk yesteryear.
Regarding the unclear use case: if you want to store large .iso-files any Raid level is fine. If you want to run VMs on it, you want to have as many IOPS as possible. For IOPS go with Raid10. You will find several threads discussing this...
Best regards
While Udo B. makes excellent points (don't know about that guarantee point - money back?) - you must realize that any SW Raid requires RAM. See also ZFS's specific requirements (IIRC; 2GB +1Gb per TB?).
I don't know your resources, but it's something to bear in mind.
I don't know your resources, but it's something to bear in mind.
Well, no, it is not about money but about reliability. ZFS will make sure it delivers unmodified data.
"Hidden data loss" is impossible with ZFS. Have you ever had "half destroyed" images read from a cheap SD card? Some storys told about "wrong data" in Excel datasheets. This is impossible with ZFS.
Yes, sure! In the past ZFS grabbed "up to" 50 percent of the system RAM for its ARC. (Newer installations default to 10 percent, I believe.) You can manually configure it to use - let's say - only 1 GiB. But as for every filesystem: the more cache the better.
Best regards
ZFS + ECC RAM. Had those half destroyed images on ZFS when accessing it with bad non-ECC RAM
Not sure what the exact definition here is of "Hidden" - but we'll all agree that data loss is possible. All data loss must have a reason, even Bitrot/Data degredation (still being studied), has a reason. So does that self-combusting knock-off SD card. Talking of Bitrot what are the stats of Bitrot on ZFS vs HW raid, IDK.
Just one thing to add; Some raid controllers may offer a decent cache + some (limited?) PLP.
I mean "without any error message reported to/from the OS".
Classic magnetic disks do not simply deliver the digital data you wrote. They calculate the data from very small analog signals with a "maximum-likelihood" --> https://en.wikipedia.org/wiki/PRML
Of course this is good enough for 99.99999999999999 percent of bytes read, so it usually is fine. But sometimes the decision of that algorithm is just wrong. The disk "believes" it is ok and sends the data to the caller. There is no way to recognize this kind of errors. Except if you have a higher level checksuming like... in ZFS (or possibly other filesystems) or in the application layer.
Please note that this describes the situation without any "real" errors like head crashes, surface damages, power supply irregularities or any other source of errors involved. It is just how it works.
For SSD/NVMe those magnetic problems have vanished... and were replaced by other technology...
Edit/added: "All in all, the error rates as observed by a CERN study on silent corruption are far higher than one in every 10^16 bits." --
https://en.wikipedia.org/wiki/Silent_data_corruption#Silent
Last edited: