Fix SSD Raid1 Issues / Suggestions?

[sappel]

Hello everyone,
I'm currently facing something that looks like disk degradation like described here: https://forum.proxmox.com/threads/ssd-on-raid1-beware-of-disk-degradation-over-the-time.19055/

I have a LSI MegaRAID 9240-8i raid adapter running with 2x 1TB semi-professional SSDs.
Now I want to replace the SSDs, but what should I go for?

Should I stick with RAID1 or switch to ZFS? RAID1 probably only with enterprise SSDs from intel or Samsung, but what about ZFS? Does ist make a difference here ir should a Samsung Pro be fine?

Happy to hear your ideas / experiences / suggestions...

 

[Klaus Steinberger]

ZFS in Mirror configuration would be nice, but be aware that Megaraid doesn't support HBA Mode, so think of cross flashing the megaraid into IT Mode (HBA). If it is an original LSI 9240, it should be possible.

 

[chrone]

Your RAID card has to support SSD TRIM on RAID array and usually only on latest generation, like LSI 3008/3108 chip or Dell PERC H730 series or above.

ZFS on Linux does not support unmap/fstrim right now. You will depend the TRIM to be performed inside a VM with virtio-scsi and discard enabled.

Use Samsung SSD Enterprise SM863a series or Intel DC S3500/3600/3700 series for best performance as they are based on MLC nand and have powerloss protection which improves synchronous write performance like a lot.

 

[sappel]

Thanks for your answers.
So if I understand correctly, TRIM won't work on my LSI 9240 in Raid Array, right? So my SSDs don't do any garbage collection.

Since I need either Raid1 or some ZFS Mirroring-magic, I should get a new raid controller + new ssds if I want to stick to classical Raid1?

The other option is to go for ZFS and flash the LSI in IT mode (or even chose the onboard ports of the Super Micro X11SSM since the SSDs are SATA and not SAS anyways). The I could go for e.g. 1x 1TB Intel DC SSD + 2x HDD and let ZFS do the mirror magic + caching on ssd? (if I spare the raid controller I might bump in a little more memory...got 32GB now, but I have another 32GB available).

What Hardware-Options for the disks have to do with this in the VM-settings of Proxmox?

I'm a bit overwhelmed with all the ZFS options. I haven't worked with ZFS yet and I'm not sure wether my Proxmox VM is the right place to start working with it...but the storage configuration we started with was not really a stroke of a genius I guess (that non-genius was me for the matter of fact..)

Perhabs it's helpful to let you guys know what I'm dealing with: 1x Win2016 Server, 1x Win2008 SBS (Exchange still in use), 1x Ubuntu-Server as a backup for our synology NAS (which is our real data graveyard). So for the 2016 I need about 500GB (1x 250GB System partition, 1x 250GB MS SQL partition) and for the Win2008 I need about 250GB on one partition for system+exchange. But I need another 2TB as backup of the NAS.

So to completly re-design the storage, could this be an idea?
SSD Cache:
1x Samsung SSD PM863a/SM863a 480GB
or
1x Intel SSD DC S3520/S3500 480GB
(all have a similar price, the S3500 is a little bit more expensive)

Storage
4x WD Red Pro 2TB in Raid 10 ZFS

 

[sappel]

Now after a few minutes of thought:
Is a ZFS Raid10 really helpful if using a SSD Cache for the ZFS? Or would sticking to a regular Raid1 be better?

 

[guletz]

If yours zfs pool have only SSDs, it is not helpful to also use another SSD as ZFS cache.

 

[sappel]

My storage re-design would end up with a HDD pool + SSD Cache

 

[guletz]

My storage re-design would end up with a HDD pool + SSD Cache

This will be OK.

 

[sappel]

What about Raid10? Any use if there is a SSD Cache or would be a Raid1 sufficient?

 

[guletz]

What about Raid10? Any use if there is a SSD Cache or would be a Raid1 sufficient?

It is hard to say for your case. A raid10 will be far better(IOPs and bandw) for any VM/container that will use any DB. An you will also think to the near future. Maybe now is OK Raid1, but how it will be after 3/6 month?

 

[sappel]

Then I might go for Raid10 since there is one MS SQL DB on one VM (+exchange and its DB stuff on another). This is getting more and more confusing

So current idea is:
4x 2TB HDD Raid10 ZFS for the VMs (replacing the 2x 960GB SSD Raid1 and 2x 1TB HDD Raid1 Data partition)
1x 180GB SSD non-raid for Proxmox system (which is currently also on the raid1 of the SSDs)
2x 2TB HDD non-raid or raid0 for internal backup (external backup exists as well)

 

[guletz]

If you ask me, I would make like this:
- only one zfs raid10, using 6x 2TB HDD(3 pairs of mirror):
zpool create NAME mirror hdd1 hdd2 mirror hdd3 hdd4 mirror hdd5 hdd6