Add SSD to RAID-10 ZFS?

[jmcorrea]

Hello!

we have 2 servers with local ZFS pools. The pools are configured with the same type of disks. They have 3 x 120GB SSD and 4 x 4TB HDD.
I set the 3 SSD as a physical RAID 5, there runs Proxmox on one partition and ZFS logs and cache on 2 other partitions. Then, with the 4 HDD I set up a RAID10:

Pool 1:

root@server-7:~# zpool status -x
all pools are healthy
root@server-7:~# zpool status
  pool: storage-pool
 state: ONLINE
  scan: scrub repaired 0B in 24h13m with 0 errors on Mon Apr 15 00:37:44 2019
config:

    NAME          STATE     READ WRITE CKSUM
    storage-pool  ONLINE       0     0     0
      mirror-0    ONLINE       0     0     0
        sdb       ONLINE       0     0     0
        sdc       ONLINE       0     0     0
      mirror-1    ONLINE       0     0     0
        sdd       ONLINE       0     0     0
        sde       ONLINE       0     0     0
    logs
      sda4        ONLINE       0     0     0
    cache
      sda5        ONLINE       0     0     0

errors: No known data errors
root@server-7:~# zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage-pool  7.25T  3.41T  3.84T         -    39%    47%  1.00x  ONLINE  -

Pool 2:

root@server-8:~# zpool status
  pool: storage-pool
 state: ONLINE
  scan: scrub repaired 0B in 183h36m with 0 errors on Sun Apr 21 16:01:40 2019
config:

    NAME          STATE     READ WRITE CKSUM
    storage-pool  ONLINE       0     0     0
      mirror-0    ONLINE       0     0     0
        sdb       ONLINE       0     0     0
        sdc       ONLINE       0     0     0
      mirror-1    ONLINE       0     0     0
        sdd       ONLINE       0     0     0
        sde       ONLINE       0     0     0
    logs
      sda4        ONLINE       0     0     0
    cache
      sda5        ONLINE       0     0     0

errors: No known data errors
root@server-8:~# zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage-pool  7.25T  4.28T  2.97T         -    47%    59%  1.00x  ONLINE  -

Now I have 4 x 1TB SSD disks and I would like to add them to the RAIDs 10, 2 SSD on each server.

Do you think this is a good way to use this SSD? Is there a better way to use them on our current configuration?

Any advice is welcome, thank you very much!

Best regards,

Juan

 

[wolfgang]

Hi,

it is generally not recommended to use ZFS in combination with Raid Cards.
ZFS has it own cache management and the transparent raid cache will interfere.

 

[jmcorrea]

Hello @wolfgang!
Yes, i read about that, that is why the SSD RAID 5 is not ZFS, is a good old LVM.
I been doing some testing and research and it seems that the way to get the most of the SSD disks is to create a new zpool for them, because if i add them to the current HDD zpool the speed i get from them will be the same as if they were HDD, is this correct?
Thank you very much!
Best regards,
Juan