Performance with write cache on hp smart array p410i ?

Discussion in 'Proxmox VE: Installation and configuration' started by fpausp, Apr 12, 2014.

  1. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    I use the hp smart array p410i without write cache because there is no battery inside...

    I did a shot test:



    ##### Proxmox - Performance on HP ML350 G6 #####


    ## without write cache ##

    pveperf

    CPU BOGOMIPS: 17062.84
    REGEX/SECOND: 854697
    HD SIZE: 19.69 GB (/dev/mapper/pve-root)
    BUFFERED READS: 540.29 MB/sec
    AVERAGE SEEK TIME: 6.64 ms
    FSYNCS/SECOND: 50.59
    DNS EXT: 35.61 ms
    DNS INT: 13.02 ms (domain.at)

    pveperf

    CPU BOGOMIPS: 17062.84
    REGEX/SECOND: 831907
    HD SIZE: 19.69 GB (/dev/mapper/pve-root)
    BUFFERED READS: 715.44 MB/sec
    AVERAGE SEEK TIME: 5.08 ms
    FSYNCS/SECOND: 52.30
    DNS EXT: 43.28 ms
    DNS INT: 17.12 ms (domain.at)


    pveperf

    CPU BOGOMIPS: 17062.84
    REGEX/SECOND: 853512
    HD SIZE: 19.69 GB (/dev/mapper/pve-root)
    BUFFERED READS: 723.48 MB/sec
    AVERAGE SEEK TIME: 4.90 ms
    FSYNCS/SECOND: 46.35
    DNS EXT: 39.57 ms
    DNS INT: 13.35 ms (domain.at)




    If write cache is enabled, how many percent could it be better ?
     
  2. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    Even if no battery is available, you can enable write cache for testing purpose. But for productive use a battery pack is recommendet.
     
  3. mir

    mir Well-Known Member
    Proxmox Subscriber

    Joined:
    Apr 14, 2012
    Messages:
    3,481
    Likes Received:
    96
    The reads is not affected by write cache but fsync greatly depends on write cache. Enabling write cache will raise your fsync per second with a factor 40 or 50 (that's 4000-5000 percent better!)
     
  4. mir

    mir Well-Known Member
    Proxmox Subscriber

    Joined:
    Apr 14, 2012
    Messages:
    3,481
    Likes Received:
    96
    If your rack is powered by a PDU battery for cache is less important.
     
  5. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    Are you sure ?

    I had enabled it but the result of pveperf was the same, I think it could not be enabled without a battery ... ?
     
  6. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    Yes, I was wrong and mixed things up.

    Drive Write Cache: If enable means the array will use the onboard cache on the hard drive. It will write the data here first and then to the disk. If the server is not procted via UPS and looses power. Data in the cache will be lost and cause data corruption on the hard drive.

    No-Battery Write Cache:If you have no battery installed or it is failed you can enable this feature to use the on-board cache of the raid controller.But if the server is not protected via UPS and looses power. Data in the cache will be lost and cause data corruption on the hard drive.

    There is another option you may try.
    To enable the cache without a battery run this command: controller slot=X modify nobatterywritecache=enable
    Then change the cache ratio to whatever you like: controller slot=X modify cacheratio=25/75

    Anyway, if you need performance it's not a bad idea to invest in a 512MB Battery-backed write cache upgrade.
     
  7. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    Is this a permanent entry ?




    I guess I can disable it with:
    Code:
    controller slot=X modify nobatterywritecache=disable
    



    I use a 256MB module, what ratio should I use ?
     
  8. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    You can choose the ratio you need, depending on your read/write performance.
    Do you have a battery backup now ?
     
  9. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    No sorry, I have to buy it ...
     
  10. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
  11. informant

    informant Member

    Joined:
    Jan 31, 2012
    Messages:
    674
    Likes Received:
    6
    Hi all, we have a P400 with BBU and 512 MB Cache on a DL360 with 6x1TB SAS HDD in Raid5. But we have only slow speed. What we can configure to get more speed?

    pveperf
    CPU BOGOMIPS: 31999.56
    REGEX/SECOND: 634252
    HD SIZE: 94.49 GB (/dev/mapper/pve-root)
    BUFFERED READS: 27.96 MB/sec
    AVERAGE SEEK TIME: 17.07 ms
    FSYNCS/SECOND: 1738.98
    DNS EXT: 55.07 ms
    DNS INT: 32.37 ms (domain.de)

    If we check speed with dd we have fast speed:

    [Info] Performing WRITE test
    1048576000 Bytes (1,0 GB) kopiert, 2,52741 s, 415 MB/s
    [Info] Finished WRITE test
    ------------------------------
    [Info] Performing READ test
    1048576000 Bytes (1,0 GB) kopiert, 1,6512 s, 635 MB/s
    [Info] Finished READ test
    ------------------------------

    What can we do, do have this speed with proxmox, too?

    regards
     
  12. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    The command line utility to check the controller configuration is “hpacucli”. Navigating using hpacucli is very straight forward.

    “ctrl all show config detail” Will give you the entire controller configuration. What does this command show ?
     
  13. informant

    informant Member

    Joined:
    Jan 31, 2012
    Messages:
    674
    Likes Received:
    6
    Hi sumsum,

    output of command:
    Code:
    Smart Array P400i in Slot 0 (Embedded)
       Bus Interface: PCI
       Slot: 0
       Serial Number: PH7AMV2725
       Cache Serial Number: PA82C0H9SV7B52
       RAID 6 (ADG) Status: Enabled
       Controller Status: OK
       Hardware Revision: Rev D
       Firmware Version: 7.24
       Rebuild Priority: High
       Expand Priority: Medium
       Surface Scan Delay: 3 secs
       Surface Scan Mode: Idle
       Wait for Cache Room: Disabled
       Surface Analysis Inconsistency Notification: Disabled
       Post Prompt Timeout: 15 secs
       Cache Board Present: True
       Cache Status: OK
       Accelerator Ratio: 50% Read / 50% Write
       Drive Write Cache: Disabled
       Total Cache Size: 512 MB
       No-Battery Write Cache: Enabled
       Cache Backup Power Source: Batteries
       Battery/Capacitor Count: 1
       Battery/Capacitor Status: OK
       SATA NCQ Supported: True
    
       Array: A
          Interface Type: SATA
          Unused Space: 0 MB
          Status: OK
    
    
    
          Logical Drive: 1
             Size: 4.5 TB
             Fault Tolerance: RAID 5
             Heads: 255
             Sectors Per Track: 32
             Cylinders: 65535
             Strip Size: 64 KB
             Status: OK
             Array Accelerator: Enabled
             Parity Initialization Status: Initialization Completed
             Unique Identifier: 600508B100103732352020202020000F
             Disk Name: /dev/cciss/c0d0
             Mount Points: /boot 510 MB
             OS Status: LOCKED
             Logical Drive Label: A24D5747PH7AMV2725     1AE1
    
          physicaldrive 1I:1:1
             Port: 1I
             Box: 1
             Bay: 1
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1JTA3
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 25
             Maximum Temperature (C): 55
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
          physicaldrive 1I:1:2
             Port: 1I
             Box: 1
             Bay: 2
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1JTDR
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 27
             Maximum Temperature (C): 56
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
          physicaldrive 1I:1:3
             Port: 1I
             Box: 1
             Bay: 3
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1XHEX
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 26
             Maximum Temperature (C): 53
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
          physicaldrive 1I:1:4
             Port: 1I
             Box: 1
             Bay: 4
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1VF2R
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 25
             Maximum Temperature (C): 56
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
          physicaldrive 2I:1:5
             Port: 2I
             Box: 1
             Bay: 5
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1XFF1
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 25
             Maximum Temperature (C): 53
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
          physicaldrive 2I:1:6
             Port: 2I
             Box: 1
             Bay: 6
             Status: OK
             Drive Type: Data Drive
             Interface Type: SATA
             Size: 1 TB
             Firmware Revision: SN02
             Serial Number:             9XG1JTEY
             Model: ATA     ST91000640NS
             SATA NCQ Capable: True
             SATA NCQ Enabled: True
             Current Temperature (C): 27
             Maximum Temperature (C): 54
             PHY Count: 1
             PHY Transfer Rate: 1.5GBPS
    
    
    regards
     
  14. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    Enable Drive Write Cache. Change Accelerator Ratio: 50% Read / 50% Write to 25%/75% and retest.
     
  15. informant

    informant Member

    Joined:
    Jan 31, 2012
    Messages:
    674
    Likes Received:
    6
    Hi sumsum,

    ok we would test it in view minutes. But if we Enable Drive Write Cache, we can lost data on power out - right?

    Can we enable Drive Write Cache with a command? if yes, can you tell it us, please?

    regards.
     
  16. informant

    informant Member

    Joined:
    Jan 31, 2012
    Messages:
    674
    Likes Received:
    6
    After ctrl slot=0 modify dwc=enable
    we have smaller speed as before:

    Code:
     pveperf
    CPU BOGOMIPS:      31999.56
    REGEX/SECOND:      641680
    HD SIZE:           94.49 GB (/dev/mapper/pve-root)
    BUFFERED READS:    16.76 MB/sec
    AVERAGE SEEK TIME: 20.81 ms
    FSYNCS/SECOND:     1857.84
    DNS EXT:           68.01 ms
    DNS INT:           38.10 ms (domain.de)
    do you have a idea? regards

    ok, after change with hpacucli ctrl slot=0 modify cacheratio=25/75 to 25/75 we have 70mb/s but not the speed of dd test, but why? Backups are slower as before change, hmm.
     
    #16 informant, Apr 13, 2014
    Last edited: Apr 13, 2014
  17. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    hmm this is strange. well then back to start and reverse the settings you make before then.

    btw. I just found in your config details, that your disks are connected with

    Code:
    PHY Transfer Rate: 1.5GBPS
    I might be wrong, but the ST91000640NS should run on PHY Transfer Rate: 6.0 GBPS ?
     
  18. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    @ sumsum

    Please take a look on that:


    Code:
    hpacucli ctrl all show config detail
    
    Smart Array P410i in Slot 0 (Embedded)
       Bus Interface: PCI
       Slot: 0
       Serial Number: 50014380145BE9A0
       Cache Serial Number: PACCQ9SXTI23
       RAID 6 (ADG) Status: Enabled
       Controller Status: OK
       Hardware Revision: C
       Firmware Version: 6.40
       Rebuild Priority: Medium
       Expand Priority: Medium
       Surface Scan Delay: 15 secs
       Surface Scan Mode: Idle
       Queue Depth: Automatic
       Monitor and Performance Delay: 60  min
       Elevator Sort: Enabled
       Degraded Performance Optimization: Disabled
       Inconsistency Repair Policy: Disabled
       Wait for Cache Room: Disabled
       Surface Analysis Inconsistency Notification: Disabled
       Post Prompt Timeout: 0 secs
       Cache Board Present: True
       Cache Status: OK
       Cache Ratio: 100% Read / 0% Write
       Drive Write Cache: Disabled
       Total Cache Size: 256 MB
       Total Cache Memory Available: 144 MB
       No-Battery Write Cache: Disabled
       Battery/Capacitor Count: 0
       SATA NCQ Supported: True
    
       Array: A
          Interface Type: SAS
          Unused Space: 0  MB
          Status: OK
          Array Type: Data
    
    
    
          Logical Drive: 1
             Size: 5.7 TB
             Fault Tolerance: 5
             Heads: 255
             Sectors Per Track: 32
             Cylinders: 65535
             Strip Size: 256 KB
             Full Stripe Size: 1792 KB
             Status: OK
             Caching:  Enabled
             Parity Initialization Status: Initialization Completed
             Unique Identifier: 600508B1001C130411F6E712697F87ED
             Disk Name: /dev/sda
             Mount Points: /boot 510 MB
             OS Status: LOCKED
             Logical Drive Label: A348049C50014380145BE9A0F4DC
             Drive Type: Data
    
          physicaldrive 1I:1:1
             Port: 1I
             Box: 1
             Bay: 1
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N11K4Q0000B419CS74
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 1I:1:2
             Port: 1I
             Box: 1
             Bay: 2
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N11K700000B419CTAY
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 1I:1:3
             Port: 1I
             Box: 1
             Bay: 3
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N11K4C0000B419CSQF
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 1I:1:4
             Port: 1I
             Box: 1
             Bay: 4
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N1953W0000M418MGA1
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 2I:1:5
             Port: 2I
             Box: 1
             Bay: 5
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N11K5S0000B419CVBE
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 2I:1:6
             Port: 2I
             Box: 1
             Bay: 6
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N10Z580000B419CTEJ
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 2I:1:7
             Port: 2I
             Box: 1
             Bay: 7
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N10Z640000B419CTEN
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
          physicaldrive 2I:1:8
             Port: 2I
             Box: 1
             Bay: 8
             Status: OK
             Drive Type: Data Drive
             Interface Type: SAS
             Size: 900.1 GB
             Rotational Speed: 10500
             Firmware Revision: 0001
             Serial Number: S0N111FZ0000B434CR9Y
             Model: SEAGATE ST900MM0026
             PHY Count: 2
             PHY Transfer Rate: 6.0Gbps, Unknown
    
    
       SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250
          Device Number: 250
          Firmware Version: RevC
          WWID: 50014380145BE9AF
          Vendor ID: PMCSIERA
          Model:  SRC 8x6G
    
    


    Any Suggestions ?
     
  19. sumsum

    sumsum Member
    Proxmox Subscriber

    Joined:
    Oct 26, 2009
    Messages:
    157
    Likes Received:
    2
    @fpausp
    first get your Battery :cool:
    but it looks good , except the Cache Ratio: 100% Read / 0% Write is a bit unbalanced - well if this is fine for your case. I dont mind.
    you are running a slightly better controller (P410i) and faster disks than informant (P400i).
     
  20. fpausp

    fpausp Member

    Joined:
    Aug 31, 2010
    Messages:
    247
    Likes Received:
    6
    Thank you sumsum, I have another problem with qmrestore posted here: http://forum.proxmox.com/threads/18307-qmrestore-error

    Has that something to do with the Raidcontroller ?

    I never had problems with proxmox until now, I normaly use Fujitsu Server with adaptec Raidcontroller ...



    P.S. should i update the Firmware of the P410i ?
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice