Performance with write cache on hp smart array p410i ?

fpausp · Apr 12, 2014

I use the hp smart array p410i without write cache because there is no battery inside...

I did a shot test:

##### Proxmox - Performance on HP ML350 G6 #####

## without write cache ##

pveperf

CPU BOGOMIPS: 17062.84
REGEX/SECOND: 854697
HD SIZE: 19.69 GB (/dev/mapper/pve-root)
BUFFERED READS: 540.29 MB/sec
AVERAGE SEEK TIME: 6.64 ms
FSYNCS/SECOND: 50.59
DNS EXT: 35.61 ms
DNS INT: 13.02 ms (domain.at)

pveperf

CPU BOGOMIPS: 17062.84
REGEX/SECOND: 831907
HD SIZE: 19.69 GB (/dev/mapper/pve-root)
BUFFERED READS: 715.44 MB/sec
AVERAGE SEEK TIME: 5.08 ms
FSYNCS/SECOND: 52.30
DNS EXT: 43.28 ms
DNS INT: 17.12 ms (domain.at)

pveperf

CPU BOGOMIPS: 17062.84
REGEX/SECOND: 853512
HD SIZE: 19.69 GB (/dev/mapper/pve-root)
BUFFERED READS: 723.48 MB/sec
AVERAGE SEEK TIME: 4.90 ms
FSYNCS/SECOND: 46.35
DNS EXT: 39.57 ms
DNS INT: 13.35 ms (domain.at)

If write cache is enabled, how many percent could it be better ?

sumsum · Apr 12, 2014

Even if no battery is available, you can enable write cache for testing purpose. But for productive use a battery pack is recommendet.

mir · Apr 12, 2014

The reads is not affected by write cache but fsync greatly depends on write cache. Enabling write cache will raise your fsync per second with a factor 40 or 50 (that's 4000-5000 percent better!)

mir · Apr 12, 2014

sumsum said:
Even if no battery is available, you can enable write cache for testing purpose. But for productive use a battery pack is recommendet.

If your rack is powered by a PDU battery for cache is less important.

fpausp · Apr 12, 2014

Even if no battery is available, you can enable write cache for testing purpose.

Are you sure ?

I had enabled it but the result of pveperf was the same, I think it could not be enabled without a battery ... ?

sumsum · Apr 12, 2014

Yes, I was wrong and mixed things up.

Drive Write Cache: If enable means the array will use the onboard cache on the hard drive. It will write the data here first and then to the disk. If the server is not procted via UPS and looses power. Data in the cache will be lost and cause data corruption on the hard drive.

No-Battery Write Cache:If you have no battery installed or it is failed you can enable this feature to use the on-board cache of the raid controller.But if the server is not protected via UPS and looses power. Data in the cache will be lost and cause data corruption on the hard drive.

There is another option you may try.
To enable the cache without a battery run this command: controller slot=X modify nobatterywritecache=enable
Then change the cache ratio to whatever you like: controller slot=X modify cacheratio=25/75

Anyway, if you need performance it's not a bad idea to invest in a 512MB Battery-backed write cache upgrade.

fpausp · Apr 13, 2014

To enable the cache without a battery run this command: controller slot=X modify nobatterywritecache=enable

Is this a permanent entry ?

I guess I can disable it with:

Code:

controller slot=X modify nobatterywritecache=disable

Then change the cache ratio to whatever you like: controller slot=X modify cacheratio=25/75

I use a 256MB module, what ratio should I use ?

sumsum · Apr 13, 2014

You can choose the ratio you need, depending on your read/write performance.
Do you have a battery backup now ?

fpausp · Apr 13, 2014

No sorry, I have to buy it ...

sumsum · Apr 13, 2014

If you have the BBU received , start doing the performance baselineing and adjust the parameters to your need. Iozone is what I use for that.Here another example :http://www.overclock.net/t/515068/tuning-a-hp-smart-array-p400-with-linux-why-tuning-really-matters
Google is full of results if you search for tuning and optimization. In general there is no recommendation for parameters. It always depend on your system configuration an the need for what you like to do.

informant · Apr 13, 2014

Hi all, we have a P400 with BBU and 512 MB Cache on a DL360 with 6x1TB SAS HDD in Raid5. But we have only slow speed. What we can configure to get more speed?

pveperf
CPU BOGOMIPS: 31999.56
REGEX/SECOND: 634252
HD SIZE: 94.49 GB (/dev/mapper/pve-root)
BUFFERED READS: 27.96 MB/sec
AVERAGE SEEK TIME: 17.07 ms
FSYNCS/SECOND: 1738.98
DNS EXT: 55.07 ms
DNS INT: 32.37 ms (domain.de)

If we check speed with dd we have fast speed:

[Info] Performing WRITE test
1048576000 Bytes (1,0 GB) kopiert, 2,52741 s, 415 MB/s
[Info] Finished WRITE test
------------------------------
[Info] Performing READ test
1048576000 Bytes (1,0 GB) kopiert, 1,6512 s, 635 MB/s
[Info] Finished READ test
------------------------------

What can we do, do have this speed with proxmox, too?

regards

sumsum · Apr 13, 2014

The command line utility to check the controller configuration is “hpacucli”. Navigating using hpacucli is very straight forward.

“ctrl all show config detail” Will give you the entire controller configuration. What does this command show ?

informant · Apr 13, 2014

Hi sumsum,

output of command:

Code:

Smart Array P400i in Slot 0 (Embedded)
   Bus Interface: PCI
   Slot: 0
   Serial Number: PH7AMV2725
   Cache Serial Number: PA82C0H9SV7B52
   RAID 6 (ADG) Status: Enabled
   Controller Status: OK
   Hardware Revision: Rev D
   Firmware Version: 7.24
   Rebuild Priority: High
   Expand Priority: Medium
   Surface Scan Delay: 3 secs
   Surface Scan Mode: Idle
   Wait for Cache Room: Disabled
   Surface Analysis Inconsistency Notification: Disabled
   Post Prompt Timeout: 15 secs
   Cache Board Present: True
   Cache Status: OK
   Accelerator Ratio: 50% Read / 50% Write
   Drive Write Cache: Disabled
   Total Cache Size: 512 MB
   No-Battery Write Cache: Enabled
   Cache Backup Power Source: Batteries
   Battery/Capacitor Count: 1
   Battery/Capacitor Status: OK
   SATA NCQ Supported: True

   Array: A
      Interface Type: SATA
      Unused Space: 0 MB
      Status: OK



      Logical Drive: 1
         Size: 4.5 TB
         Fault Tolerance: RAID 5
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 65535
         Strip Size: 64 KB
         Status: OK
         Array Accelerator: Enabled
         Parity Initialization Status: Initialization Completed
         Unique Identifier: 600508B100103732352020202020000F
         Disk Name: /dev/cciss/c0d0
         Mount Points: /boot 510 MB
         OS Status: LOCKED
         Logical Drive Label: A24D5747PH7AMV2725     1AE1

      physicaldrive 1I:1:1
         Port: 1I
         Box: 1
         Bay: 1
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1JTA3
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 25
         Maximum Temperature (C): 55
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

      physicaldrive 1I:1:2
         Port: 1I
         Box: 1
         Bay: 2
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1JTDR
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 27
         Maximum Temperature (C): 56
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

      physicaldrive 1I:1:3
         Port: 1I
         Box: 1
         Bay: 3
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1XHEX
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 26
         Maximum Temperature (C): 53
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

      physicaldrive 1I:1:4
         Port: 1I
         Box: 1
         Bay: 4
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1VF2R
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 25
         Maximum Temperature (C): 56
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

      physicaldrive 2I:1:5
         Port: 2I
         Box: 1
         Bay: 5
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1XFF1
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 25
         Maximum Temperature (C): 53
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

      physicaldrive 2I:1:6
         Port: 2I
         Box: 1
         Bay: 6
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Firmware Revision: SN02
         Serial Number:             9XG1JTEY
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 27
         Maximum Temperature (C): 54
         PHY Count: 1
         PHY Transfer Rate: 1.5GBPS

regards

sumsum · Apr 13, 2014

Enable Drive Write Cache. Change Accelerator Ratio: 50% Read / 50% Write to 25%/75% and retest.

informant · Apr 13, 2014

Hi sumsum,

ok we would test it in view minutes. But if we Enable Drive Write Cache, we can lost data on power out - right?

Can we enable Drive Write Cache with a command? if yes, can you tell it us, please?

regards.

informant · Apr 13, 2014

After ctrl slot=0 modify dwc=enable
we have smaller speed as before:

Code:

 pveperf
CPU BOGOMIPS:      31999.56
REGEX/SECOND:      641680
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    16.76 MB/sec
AVERAGE SEEK TIME: 20.81 ms
FSYNCS/SECOND:     1857.84
DNS EXT:           68.01 ms
DNS INT:           38.10 ms (domain.de)

do you have a idea? regards

ok, after change with hpacucli ctrl slot=0 modify cacheratio=25/75 to 25/75 we have 70mb/s but not the speed of dd test, but why? Backups are slower as before change, hmm.

sumsum · Apr 13, 2014

hmm this is strange. well then back to start and reverse the settings you make before then.

btw. I just found in your config details, that your disks are connected with

Code:

PHY Transfer Rate: 1.5GBPS

I might be wrong, but the ST91000640NS should run on PHY Transfer Rate: 6.0 GBPS ?

fpausp · Apr 13, 2014

@ sumsum

Please take a look on that:

Code:

hpacucli ctrl all show config detail

Smart Array P410i in Slot 0 (Embedded)
   Bus Interface: PCI
   Slot: 0
   Serial Number: 50014380145BE9A0
   Cache Serial Number: PACCQ9SXTI23
   RAID 6 (ADG) Status: Enabled
   Controller Status: OK
   Hardware Revision: C
   Firmware Version: 6.40
   Rebuild Priority: Medium
   Expand Priority: Medium
   Surface Scan Delay: 15 secs
   Surface Scan Mode: Idle
   Queue Depth: Automatic
   Monitor and Performance Delay: 60  min
   Elevator Sort: Enabled
   Degraded Performance Optimization: Disabled
   Inconsistency Repair Policy: Disabled
   Wait for Cache Room: Disabled
   Surface Analysis Inconsistency Notification: Disabled
   Post Prompt Timeout: 0 secs
   Cache Board Present: True
   Cache Status: OK
   Cache Ratio: 100% Read / 0% Write
   Drive Write Cache: Disabled
   Total Cache Size: 256 MB
   Total Cache Memory Available: 144 MB
   No-Battery Write Cache: Disabled
   Battery/Capacitor Count: 0
   SATA NCQ Supported: True

   Array: A
      Interface Type: SAS
      Unused Space: 0  MB
      Status: OK
      Array Type: Data



      Logical Drive: 1
         Size: 5.7 TB
         Fault Tolerance: 5
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 65535
         Strip Size: 256 KB
         Full Stripe Size: 1792 KB
         Status: OK
         Caching:  Enabled
         Parity Initialization Status: Initialization Completed
         Unique Identifier: 600508B1001C130411F6E712697F87ED
         Disk Name: /dev/sda
         Mount Points: /boot 510 MB
         OS Status: LOCKED
         Logical Drive Label: A348049C50014380145BE9A0F4DC
         Drive Type: Data

      physicaldrive 1I:1:1
         Port: 1I
         Box: 1
         Bay: 1
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N11K4Q0000B419CS74
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 1I:1:2
         Port: 1I
         Box: 1
         Bay: 2
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N11K700000B419CTAY
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 1I:1:3
         Port: 1I
         Box: 1
         Bay: 3
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N11K4C0000B419CSQF
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 1I:1:4
         Port: 1I
         Box: 1
         Bay: 4
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N1953W0000M418MGA1
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 2I:1:5
         Port: 2I
         Box: 1
         Bay: 5
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N11K5S0000B419CVBE
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 2I:1:6
         Port: 2I
         Box: 1
         Bay: 6
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N10Z580000B419CTEJ
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 2I:1:7
         Port: 2I
         Box: 1
         Bay: 7
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N10Z640000B419CTEN
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 2I:1:8
         Port: 2I
         Box: 1
         Bay: 8
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 900.1 GB
         Rotational Speed: 10500
         Firmware Revision: 0001
         Serial Number: S0N111FZ0000B434CR9Y
         Model: SEAGATE ST900MM0026
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown


   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250
      Device Number: 250
      Firmware Version: RevC
      WWID: 50014380145BE9AF
      Vendor ID: PMCSIERA
      Model:  SRC 8x6G

Any Suggestions ?

sumsum · Apr 13, 2014

@fpausp
first get your Battery

but it looks good , except the Cache Ratio: 100% Read / 0% Write is a bit unbalanced - well if this is fine for your case. I dont mind.
you are running a slightly better controller (P410i) and faster disks than informant (P400i).

fpausp · Apr 13, 2014

Thank you sumsum, I have another problem with qmrestore posted here: http://forum.proxmox.com/threads/18307-qmrestore-error

Has that something to do with the Raidcontroller ?

I never had problems with proxmox until now, I normaly use Fujitsu Server with adaptec Raidcontroller ...

P.S. should i update the Firmware of the P410i ?

Performance with write cache on hp smart array p410i ?

Renowned Member

Renowned Member

Famous Member

Famous Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member

Renowned Member