Quick question, hopefully

[Hsilamot]

So, i'm setting up my first proxmox cluster of servers.

The first one, i configured the HP Raid card with HBA and did a ZFS pool "RAID5", which i'm starting to think now that i have some VM's running on it as a mistake.

I'm using ST16000NM001G as HDD's (4 drives per server), i have HP DL360G9 servers, with some load i can get a I/O Wait of about 40% and server loads of up to 130.0, so yeah...

So, my question is, i'm planning on using HA so basically 2 servers or even 3 with the same data. My normal tought to speed up the IO of the hard drives would be to do a RAID-Level (on HP's Hardware) 1+0, so basically having up to 2 drive failure tolerance (yes, on only one side of the "1" part of the raid), or i can go extreme and do a RAID 0 on all 4 drives, and trust the HA to keep the services up in case of a drive failure, which, in this case if a drive failure were to occour i would need to re-make the RAID Array on that server and wait for the HA to re-replicate to that server the whole pool, that's the downside.

So, my question is "What do you think of this idea? i'm nuts? should i go for the traditional RAID10 approach?"

Thank you in advance =)

 

[Dunuin]

A striped mirror (Raid10) should double your IOPS. You can also add a mirror of small (for example 200GB) enterprise SSDs as "special devices" so the HDDs only need to store the data and all metadata is stored on the SSDs. That way the HDDs get hit by less IO. And you could enable the relatime (zfs set atime=on YourPool && zfs set relatime=on YourPool) attribute for your ZFS pool so not every read will cause a additional write.

 

[Hsilamot]

yes, i'm aware of more IOPS, that's why i realized that RAID5 was a mistake, but my choice now is either RAID10 or RAID0, basically do i take the risk with HA? and trust HA, or do i go double-safe-net

On your comment, about the metadata, i'm planning on putting a PCIe SSD Adapter card with an NVMe disk, or 2 of 1TB each, this "Special devices" is an option in ZFS pools? like to store all metadata and high-trarffic info?

thanks for the reply btw

 

[Dunuin]

yes, i'm aware of more IOPS, that's why i realized that RAID5 was a mistake, but my choice now is either RAID10 or RAID0, basically do i take the risk with HA? and trust HA, or do i go double-safe-net

On your comment, about the metadata, i'm planning on putting a PCIe SSD Adapter card with an NVMe disk, or 2 of 1TB each, this "Special devices" is an option in ZFS pools? like to store all metadata and high-trarffic info?

thanks for the reply btw

'Special device' is a special type to vdev. See here: https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954

Also keep in mind that this is no cache. If your special device SSDs fail you also loose your data on the HDDs. So you want tha same reliability for them as for your HDDs.

 

[Hsilamot]

i will check the post you sent me, thanks, and yes i was thinking along the day that i probably want at least RAID1 on that special device since it's going to contain the metadata and yes i was assuming that loosing that metadata was pretty much loosing the index of my hard disk,.