Add SSD to RAID-10 ZFS?

Discussion in 'Proxmox VE: Installation and configuration' started by jmcorrea, Apr 22, 2019.

Tags:
  1. jmcorrea

    jmcorrea New Member
    Proxmox Subscriber

    Joined:
    Jul 16, 2018
    Messages:
    15
    Likes Received:
    0
    Hello!

    we have 2 servers with local ZFS pools. The pools are configured with the same type of disks. They have 3 x 120GB SSD and 4 x 4TB HDD.
    I set the 3 SSD as a physical RAID 5, there runs Proxmox on one partition and ZFS logs and cache on 2 other partitions. Then, with the 4 HDD I set up a RAID10:

    Pool 1:
    Code:
    root@server-7:~# zpool status -x
    all pools are healthy
    root@server-7:~# zpool status
      pool: storage-pool
     state: ONLINE
      scan: scrub repaired 0B in 24h13m with 0 errors on Mon Apr 15 00:37:44 2019
    config:
    
        NAME          STATE     READ WRITE CKSUM
        storage-pool  ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            sdb       ONLINE       0     0     0
            sdc       ONLINE       0     0     0
          mirror-1    ONLINE       0     0     0
            sdd       ONLINE       0     0     0
            sde       ONLINE       0     0     0
        logs
          sda4        ONLINE       0     0     0
        cache
          sda5        ONLINE       0     0     0
    
    errors: No known data errors
    root@server-7:~# zpool list
    NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
    storage-pool  7.25T  3.41T  3.84T         -    39%    47%  1.00x  ONLINE  -
    
    Pool 2:
    Code:
    root@server-8:~# zpool status
      pool: storage-pool
     state: ONLINE
      scan: scrub repaired 0B in 183h36m with 0 errors on Sun Apr 21 16:01:40 2019
    config:
    
        NAME          STATE     READ WRITE CKSUM
        storage-pool  ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            sdb       ONLINE       0     0     0
            sdc       ONLINE       0     0     0
          mirror-1    ONLINE       0     0     0
            sdd       ONLINE       0     0     0
            sde       ONLINE       0     0     0
        logs
          sda4        ONLINE       0     0     0
        cache
          sda5        ONLINE       0     0     0
    
    errors: No known data errors
    root@server-8:~# zpool list
    NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
    storage-pool  7.25T  4.28T  2.97T         -    47%    59%  1.00x  ONLINE  -
    
    Now I have 4 x 1TB SSD disks and I would like to add them to the RAIDs 10, 2 SSD on each server.

    Do you think this is a good way to use this SSD? Is there a better way to use them on our current configuration?

    Any advice is welcome, thank you very much!

    Best regards,

    Juan
     
  2. wolfgang

    wolfgang Proxmox Staff Member
    Staff Member

    Joined:
    Oct 1, 2014
    Messages:
    4,598
    Likes Received:
    306
    Hi,

    it is generally not recommended to use ZFS in combination with Raid Cards.
    ZFS has it own cache management and the transparent raid cache will interfere.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. jmcorrea

    jmcorrea New Member
    Proxmox Subscriber

    Joined:
    Jul 16, 2018
    Messages:
    15
    Likes Received:
    0
    Hello @wolfgang!
    Yes, i read about that, that is why the SSD RAID 5 is not ZFS, is a good old LVM.
    I been doing some testing and research and it seems that the way to get the most of the SSD disks is to create a new zpool for them, because if i add them to the current HDD zpool the speed i get from them will be the same as if they were HDD, is this correct?
    Thank you very much!
    Best regards,
    Juan
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice