[SOLVED] Perfomance Issues with HP SE316M1 + P410 incl. BBU

C

cymech

New Member
Nov 24, 2013
13
0
1
Hey Proxmox Community.

We have bad i/O HDD Perfomance with our HP Servers with P410 and BBU. I Hope somebody can help us.

We use HP SE316M1 + P410 incl. BBU with WD Raid Edtion (WD 1TB WD1003FBYZ RE4 SA3)

When we have a fresh installed Proxmox the Values are very better and i think "normal" but when we have Customers on the Server the Performance are very terrible.

PVEPERF

PHP: 
pveperf

CPU BOGOMIPS:      93861.92
REGEX/SECOND:      916805
HD SIZE:           3511.64 GB (/dev/mapper/pve-root)
BUFFERED READS:    6.23 MB/sec
AVERAGE SEEK TIME: 88.12 ms
FSYNCS/SECOND:     91.18
DNS EXT:           44.15 ms
DNS INT:           28.12 ms (rack.ffm.internal.xx.de)

Mount:

PHP: 
mountsysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=8242983,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=6596028k,mode=755)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro,barrier=1,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=13192040k)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/sda2 on /boot type ext2 (rw,relatime,errors=continue)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
none on /sys/kernel/config type configfs (rw,relatime)
beancounter on /proc/vz/beancounter type cgroup (rw,relatime,blkio,name=beancounter)
container on /proc/vz/container type cgroup (rw,relatime,freezer,devices,name=container)
fairsched on /proc/vz/fairsched type cgroup (rw,relatime,cpuacct,cpu,cpuset,name=fairsched)

PVEVERSION

PHP: 
 pveversion -v

proxmox-ve-2.6.32: 3.4-156 (running kernel: 2.6.32-39-pve)
pve-manager: 3.3-16 (running version: 3.3-16/97194ea4)
pve-kernel-2.6.32-39-pve: 2.6.32-157
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-1
pve-cluster: 3.0-15
qemu-server: 3.3-9
pve-firmware: 1.1-3
libpve-common-perl: 3.0-23
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-25
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8vzctl: 4.6-1
pve1vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.2-11
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

PHP: 
 hpacucli ctrl all show config detail

Smart Array P410 in Slot 2
   Bus Interface: PCI
   Slot: 2
   Serial Number: PACCRID112303Y1
   Cache Serial Number: PBCDH0CRH1N9I7
   RAID 6 (ADG) Status: Disabled
   Controller Status: OK
   Hardware Revision: C
   Firmware Version: 6.40
   Rebuild Priority: Medium
   Expand Priority: Medium
   Surface Scan Delay: 15 secs
   Surface Scan Mode: Idle
   Queue Depth: Automatic
   Monitor and Performance Delay: 60  min
   Elevator Sort: Enabled
   Degraded Performance Optimization: Disabled
   Inconsistency Repair Policy: Disabled
   Wait for Cache Room: Disabled
   Surface Analysis Inconsistency Notification: Disabled
   Post Prompt Timeout: 0 secs
   Cache Board Present: True
   Cache Status: OK
   Cache Ratio: 25% Read / 75% Write
   Drive Write Cache: Disabled
   Total Cache Size: 512 MB
   Total Cache Memory Available: 400 MB
   No-Battery Write Cache: Enabled
   Cache Backup Power Source: Capacitors
   Battery/Capacitor Count: 1
   Battery/Capacitor Status: OK
   SATA NCQ Supported: True

PHP: 
Array: A
      Interface Type: SATA
      Unused Space: 0  MB
      Status: OK
      Array Type: Data



      Logical Drive: 1
         Size: 1.8 TB
         Fault Tolerance: RAID 1+0
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 65535
         Strip Size: 256 KB
         Full Stripe Size: 512 KB
         Status: OK
         Caching:  Enabled
         Unique Identifier: 600508B1001C6C2978CB41671EF5BC36
         Disk Name: /dev/sda
         Mount Points: /boot 243 MB
         OS Status: LOCKED
         Logical Drive Label: ACA6731APACCRID112303Y1BF8D
         Mirror Group 0:
            physicaldrive 2I:1:1 (port 2I:box 1:bay 1, SATA, 1 TB, OK)
            physicaldrive 2I:1:2 (port 2I:box 1:bay 2, SATA, 1 TB, OK)
         Mirror Group 1:
            physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SATA, 1 TB, OK)
            physicaldrive 2I:1:4 (port 2I:box 1:bay 4, SATA, 1 TB, OK)
         Drive Type: Data

      physicaldrive 2I:1:1
         Port: 2I
         Box: 1
         Bay: 1
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Rotational Speed: 7200
         Firmware Revision: 01.01V03
         Serial Number:      WD-WCAW30CS0HC6
         Model: ATA     WDC WD1003FBYZ-0
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 26
         Maximum Temperature (C): 39
         PHY Count: 1
         PHY Transfer Rate: 3.0Gbps

      physicaldrive 2I:1:2
         Port: 2I
         Box: 1
         Bay: 2
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Rotational Speed: 7200
         Firmware Revision: 01.01V03
         Serial Number:      WD-WCAW35HE6D1P
         Model: ATA     WDC WD1003FBYZ-0
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 26
         Maximum Temperature (C): 37
         PHY Count: 1
         PHY Transfer Rate: 3.0Gbps

      physicaldrive 2I:1:3
         Port: 2I
         Box: 1
         Bay: 3
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Rotational Speed: 7200
         Firmware Revision: 01.01V03
         Serial Number:      WD-WCAW34DL5X9X
         Model: ATA     WDC WD1003FBYZ-0
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 27
         Maximum Temperature (C): 37
         PHY Count: 1
         PHY Transfer Rate: 3.0Gbps

      physicaldrive 2I:1:4
         Port: 2I
         Box: 1
         Bay: 4
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Rotational Speed: 7200
         Firmware Revision: 01.01V03
         Serial Number:      WD-WCAW35HE6DHX
         Model: ATA     WDC WD1003FBYZ-0
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 27
         Maximum Temperature (C): 38
         PHY Count: 1
         PHY Transfer Rate: 3.0Gbps



   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250
      Device Number: 250
      Firmware Version: RevC
      WWID: 500143801466A26F
      Vendor ID: PMCSIERA
      Model:  SRC 8x6G

We use older Tower Server with a LSI9450se 256MB Cache and BBU without any Problems.

We hope somebody can help us. Thanks and regards
 
Last edited:
Sumsum , thanks for reply.

can you tell me what controller u now use?

I think about a P420? And what is with the Backplane. When we use another controller can we still use the Backplane that is build in the server?
 
Anything better than P410 works. Example P440. Or a third partie controller from LSI. Check the HP Site, what Would fit for your Server. You can keep the backplane in Most of the cases.
 
try to change
Drive Write Cache: Disabled
to
Drive Write Cache: Enabled
by using command like this (from root):
hpacucli controller slot=2 modify dwc=enable
 
blowfish , ok we try it.

what is with a HP P800 Controller is it much better than a p410?
 
pveperf /data/raid10/
CPU BOGOMIPS: 102396.60
REGEX/SECOND: 878886
HD SIZE: 3300.54 GB (/dev/sdb)
BUFFERED READS: 710.12 MB/sec
AVERAGE SEEK TIME: 5.38 ms
FSYNCS/SECOND: 3586.59
DNS EXT: 256.99 ms
DNS INT: 2.61 ms (****.****)



Smart Array P410 in Slot 1
Bus Interface: PCI
Slot: 1
Serial Number: *****
Cache Serial Number: ******
RAID 6 (ADG) Status: Enabled
Controller Status: OK
Hardware Revision: C
Firmware Version: 6.48
Rebuild Priority: Medium
Expand Priority: Medium
Surface Scan Delay: 15 secs
Surface Scan Mode: Idle
Queue Depth: Automatic
Monitor and Performance Delay: 60 min
Elevator Sort: Enabled
Degraded Performance Optimization: Disabled
Inconsistency Repair Policy: Disabled
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 0 secs
Cache Board Present: True
Cache Status: OK
Cache Ratio: 50% Read / 50% Write
Drive Write Cache: Enabled
Total Cache Size: 512 MB
Total Cache Memory Available: 400 MB
No-Battery Write Cache: Enabled
Battery/Capacitor Count: 0
SATA NCQ Supported: True





Smart Array P410 in Slot 1 (sn: PACCR9VYEIZJ )


array A (SATA, Unused Space: 0 MB)




logicaldrive 1 (465.7 GB, RAID 1, OK)


physicaldrive 1I:1:13 (port 1I:box 1:bay 13, SATA, 500 GB, OK)
physicaldrive 1I:1:14 (port 1I:box 1:bay 14, SATA, 500 GB, OK)


array B (SAS, Unused Space: 0 MB)




logicaldrive 2 (3.3 TB, RAID 1+0, OK)


physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 600 GB, OK)
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 600 GB, OK)
physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 600 GB, OK)
physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 600 GB, OK)
physicaldrive 1I:1:5 (port 1I:box 1:bay 5, SAS, 600 GB, OK)
physicaldrive 1I:1:6 (port 1I:box 1:bay 6, SAS, 600 GB, OK)
physicaldrive 1I:1:7 (port 1I:box 1:bay 7, SAS, 600 GB, OK)
physicaldrive 1I:1:8 (port 1I:box 1:bay 8, SAS, 600 GB, OK)
physicaldrive 1I:1:9 (port 1I:box 1:bay 9, SAS, 600 GB, OK)
physicaldrive 1I:1:10 (port 1I:box 1:bay 10, SAS, 600 GB, OK)
physicaldrive 1I:1:11 (port 1I:box 1:bay 11, SAS, 600 GB, OK)
physicaldrive 1I:1:12 (port 1I:box 1:bay 12, SAS, 600 GB, OK)


Enclosure SEP (Vendor ID HP, Model DL18xG6BP) 248 (WWID: *******, Port: 1I, Box: 1)


Expander 250 (WWID: 50014380079B1BB0, Port: 1I, Box: 1)


SEP (Vendor ID PMCSIERA, Model SRC 8x6G) 249 (WWID: ******)


this test was runned on load
 
wow, with "on load" you mean there are vms running? okay the difference is that you use SAS HDDs and we use SATA. But i dont see why we dont became speed on our Raid when it is on load.

We have now 5 Nodes with this Problem.. it is a horror =/

i enable the "Drive write cache":

PHP: 
pveperf
CPU BOGOMIPS:      93862.64
REGEX/SECOND:      985019
HD SIZE:           1759.24 GB (/dev/mapper/pve-root)
BUFFERED READS:    7.28 MB/sec
AVERAGE SEEK TIME: 45.48 ms
FSYNCS/SECOND:     0.28
DNS EXT:           43.14 ms
DNS INT:           28.24 ms (rack.ffm.internal.myvirtualserver.de)
 
Last edited:
yes, this host is on load

take another test on different node, attached screenshots
tested raid10 on 4 1tb sata hdd
123.png
456.png
test.png

this node also on light load and also dl180g6 with p410
 
thanks for reply blowfish

wow, fantastic performance. Can you show me your mount Options and pveversion? What SATA HDDs u use?

i cant see what is our problem..
 
Last edited:
your config:
RAID 6 (ADG) Status: Disabled

my config:
RAID 6 (ADG) Status: Enabled

this means my cache key differents than yours, try to use different cache key, but maybe I`m wrong.

http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/RAID-and-License-Key/td-p/5433167
 
Okay, but we dont use Raid 6? it is the SE316M1 with four 3.5 bays. i think a raid 6 with 4 HDDs is not possible.
 
this key not only activate raid6 but give some another features


  • RAID 6 (ADG).
  • RAID 60.
  • Advanced Capacity Expansion.
  • Mirror splitting and recombining in offline mode.
  • Drive Erase.
  • Performance optimization for video on demand.
 
Hey,
what kernel and pveversions do you use?
Activating the advanced functionality does not solve this problem.

Code: 
pveversion -v
Code: 
uname -a

Please also post your /etc/fstab.

Code: 
cat /etc/fstab
Code: 
lvdisplay


Best regards
Phinitris
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back