Suggestion to move forward with 12 disks storage for a PVE system

[mauro2306]

Hi everyone,

i know, there is a lot of intel around that topic, but i preferred to open a thread and have an ad-hoc analysis on my case.
I opened some thread in the past because i was building a server using Proxmox to host a router distribution (opnsense), an Openmediavault VM to handle different services such as Plex, Nextcloud and so on, and play with random Windows or Linux VM from time to time.
Back then, i wasn't sure to either use the HP SmartArray Hardware RAID card or ZFS. I tried both, and eventually, got better performance overall with Hardware RAID.
https://forum.proxmox.com/threads/cannot-store-on-zfs-raidz-volume-out-of-space.132035/#post-581295

The disks i was using for Data storage were Seagate 2.5" Barracuda 5TB, and as you can see in the thread, they were consumer drives and what needed to happen happened, i got so many drive failure in a single week that the replacement drives didn't came in time and i lost the RAID.

Not a big deal as i was running frequent backups over another server with Proxmox Backup server, but i didn't wanted to restart on the basis of that server, and invested in a 3.5" drives server plus 2x 2.5" in the rear Dell PowerEdge R730XD with proper enterprise classe drives, 12 14TB from HGST in 4Kn and 2x 7.68TB SAS SSD from Samsung.

I am still puzzled if i should go with hardware RAID using the H730p controller or if i should use ZFS with a HBA300 (that i have as well).
I know the benefit from ZFS, but i am wondering since my data drive will be filled by the VM virtual disk anyway, will the resilver really be faster in ZFS than Hardware RAID in this case ?

I wanted to go with a RAID 10 for the data storage on the 12 drives, plus a Raid 1 for the 2x SSD.

What is your opinion on this ? What i am looking for is a good balance between performance (i have 10GB cards in both the PVE and Proxmox Backup Server hosts) and reliability.

Let me know if you need some other information.
thanks.

 

[Dunuin]

The disks i was using for Data storage were Seagate 2.5" Barracuda 5TB, and as you can see in the thread, they were consumer drives and what needed to happen happened, i got so many drive failure in a single week that the replacement drives didn't came in time and i lost the RAID.

Those are SMR HDDs. Don't use SMR HDDs for anything if possible, especially not for server workloads or ZFS. CMR HDDs only exists up to 2.4TB in 2.5" form factor. Once the CMR-cache is full those SMR HDDs will totally suck and can't keep up writing.

 

[mauro2306]

Those are SMR HDDs. Don't use SMR HDDs for anything if possible, especially not for server workloads or ZFS. CMR HDDs only exists up to 2.4TB in 2.5" form factor. Once the CMR-cache is full those SMR HDDs will totally suck and can't keep up writing.

I know, but these were in the old server, no worries, as said, the new server is composed of :
- 12 disks HGST SAS 7200rpm of 14TB
- 2 SSD Samsung SAS 7.68TB

I want to know how would you go forward with RAID 10 on the 12 disks, ZFS or Hardware RAID knowing the setup i described and my use case of virtual disks that takes most of the RAID10 storage. Is ZFS still relevant, will the resilver take account of the fact that i did not filled in completely the RAW volumes or the fact that the RAW volumes take almost the whole disk will dictate a long resilvering like a hardware RAID ?

Thanks.

 

[mauro2306]

Anyone ? I know that the subject has been addressed numerous time, but things evolve on one side, and on the second, i couldn't find an exact use case as the one i try to address, so.
Thanks.