I am rebuilding a new proxmox server. 4 4TB sata drives. I plan on putting 3 in raid1 zfs mirror and using 4th for backups. Server has 32gb of ram with quad core CPU. I could likely fit a single 2.5 ssd in the case yet. Limited by 4 hot swap bays in supermicro 1u case. Would adding a single ssd as cache improve performance much? My busiest container is very disk I/o intensive, mainly writes. Its currently running on proxmox 3.x box with a couple drives in software raid1 and ext4 file system. Am I going to see much worse performance on zfs?
ZFS Raid Performance
- Thread starter matthew
- Start date
If your requirements is support for i/o intensive applications then raidz1 is a bad choice. The best choice guarantying the highest i/o rate is raid 10 (2 or more stripped mirrors). The more stripped mirrors the higher i/o.
Matthew said he was planning on putting three disks into a mirror (what most of the world, but not ZFS, calls RAID 1), not into RAIDZ. ZFS supports n-way mirrors, with n >= 2. I guess it might perform better than RAIDZ, though certainly not as well as two pairs of mirrored disks striped. It would, of course, be very wasteful of capacity.
The server will be located a good distance away and I like idea that raid1 with 3 disks will mean even if one disk in array fails I still will not have to be in any rush to replace it. As I understand looking at proxmox docs zfs supports raid-1 and raid-z1 among others. Z1 I am guessing is similiar to raid5 and raid-1 I am thinking is raid1 unless I am mistaken? I think raid1 will perform better? If I had 6 bays I would do raid10 with a spare and one drive for backups but I only have 4 bays. I think I can fit one or maybe two 2.5" ssds inside the case and was just wandering if that could be used as cache to increase I/o performance? Smaller Intel 100gb or less ssds are not to expensive.
ZFS doesn't, strictly speaking, have the traditional numbered RAID levels (0, 1, 5, 6, 10, etc.). It instead has stripes, mirrors, and RAIDZ(1,2,3). Stripes are similar to RAID0, mirrors to RAID1, and RAIDZ1, 2, and 3 are comparable to RAID5, 6, and (nonexistent triple-parity RAID). Apparently the proxmox developers have valued familiarity over accuracy in writing the installer code, and referred to stripes and mirrors by their non-ZFS analogs. So, to make a long answer shorter, what the installer calls RAID-1 isn't truly RAID1, but it's close enough.
As to cache devices, I'm not familiar enough with ZFS on Linux to say. On FreeBSD, they don't generally recommend a read cache (L2ARC) unless you've got lots of RAM--the number I see used is at least 64 GB. If you're running VMs on the machine as well, add the RAM used by those. For a write cache (SLOG or ZIL), the right device will help if your workload involves lots of synchronous writes, otherwise it won't. In either case, you'd want to use an SSD whose performance characteristics are appropriate for that job--most consumer SSDs aren't.
As to cache devices, I'm not familiar enough with ZFS on Linux to say. On FreeBSD, they don't generally recommend a read cache (L2ARC) unless you've got lots of RAM--the number I see used is at least 64 GB. If you're running VMs on the machine as well, add the RAM used by those. For a write cache (SLOG or ZIL), the right device will help if your workload involves lots of synchronous writes, otherwise it won't. In either case, you'd want to use an SSD whose performance characteristics are appropriate for that job--most consumer SSDs aren't.
Self-healing. In a traditional RAID1 mirror you don't know which drive is right and which is wrong on a corrupted block.
RAID1 is simply defined to mirror data on several disk. ZFS checksums are an internal implementation detail, so IMHO it is 100% correct to
use the term RAID1.
It's a fair question, and one for which I probably should have had a better answer in mind when I made that statement. The best answer I can give is that all the ZFS docs, whether from Oracle, or FreeBSD, or open-zfs.org, or Illumos, or zfsonlinux.org, all use the term "mirror", not "RAID1". I have to believe that there was a reason for using that term, rather than the already-familiar term of RAID1.
As a practical matter, it's very easy to confuse RAID1 with RAIDZ1, as we've seen in this very thread in @mir's post.
So:
raidz1 => raid with one parity
raidz2 => raid with two parity
raidz3 => raid with three parity
?
raidz1 => raid with one parity
raidz2 => raid with two parity
raidz3 => raid with three parity
?
Correct. RAIDZ1 is comparable to RAID5, and RAIDZ2 is comparable to RAID6. I don't believe there's anything analogous to RAIDZ3 in the traditional RAID levels, but it has three disks' worth of parity. They will tolerate the failure of one, two, or three disks, respectively, without data loss, assuming the remaining disks continue working and have no undetected data errors.
Another source that seems to have some good information on L2ARC and SLOG devices is http://louwrentius.com/things-you-should-consider-when-building-a-zfs-nas.html. I don't agree with everything he has to say there, but the explanation of those devices seems pretty reasonable.