SSD Speed was slow

[informant]

Hi, i have create a RAID 10 with 4x SSD 512 GB SAMSUNG HDDs. I use a Hardware Raid with Cache and BBU.
If i make with original fstab entry:

/dev/pve/root / ext4 errors=remount-ro 0 1
/dev/pve/data /var/lib/vz ext4 defaults 0 1
/dev/pve/swap none swap sw 0 0

pveperf /var/lib/vz
CPU BOGOMIPS:      48001.12
REGEX/SECOND:      865523
HD SIZE:           797.85 GB (/dev/mapper/pve-data)
BUFFERED READS:    266.64 MB/sec
AVERAGE SEEK TIME: 0.19 ms
FSYNCS/SECOND:     2935.39
DNS EXT:           68.47 ms
DNS INT:           57.05 ms

Speed is low.

If i change fstab to:

/dev/pve/root / ext4 relatime,nodelalloc,barrier=0,errors=remount-ro 0 1
/dev/pve/data /var/lib/vz ext4 relatime,nodelalloc,barrier=0 1
/dev/pve/swap none swap sw 0 0

and restart host and start again:

pveperf /var/lib/vz
CPU BOGOMIPS:      48000.40
REGEX/SECOND:      870037
HD SIZE:           797.85 GB (/dev/mapper/pve-data)
BUFFERED READS:    267.28 MB/sec
AVERAGE SEEK TIME: 0.19 ms
FSYNCS/SECOND:     3079.00
DNS EXT:           60.51 ms
DNS INT:           52.61 ms

Speed are slow too?

Why it was slow? Any Ideas?

best regards

 

[mir]

What is the brand and model of your hardware raid?

 

[Phinitris]

Hello,
I don't really know if it's a Proxmox problem. Please try to boot a Debian live CD or rescue system and mount your filesystem. Then
try to directly test speeds using dd.
It could be, that Proxmox's pveperf util is only running on one CPU core and therefore limited. Not quite sure about that.

dd if=/dev/zero of=/mnt/yourpve-data-mount-point bs=1G count=1 oflag=dsync

Best regards
Phinitris

 

[informant]

Hi, it´s a HP P400 Controller. Test two was a P800 Controller. Same issu.

if i start

dd if=/dev/zero of=testfile bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 1,08144 s, [B]970 MB/s[/B]

dd if=/dev/zero of=testfile bs=1G count=1  oflag=dsync
1+0 Datensätze ein
1+0 Datensätze aus
1073741824 Bytes (1,1 GB) kopiert, 6,96944 s, 154 MB/s

dd if=/dev/zero of=testfile bs=1G count=1
1+0 Datensätze ein
1+0 Datensätze aus
1073741824 Bytes (1,1 GB) kopiert, 1,84356 s, 582 MB/s

regards

 

[mir]

It seems HP P400 is only supporting SATA 1 (1.5 Gb) so maybe you get the performance which it is capable of?

 

[informant]

Hi, of curse, but 1.5Gb are not 267.28 MB/sec only in pref, if @Phinitris mean, its a proxmox problem with one kernel on test - it is right?

what you mean, do i must use original fstab entry or my post modded entry for best performance?

regards

 

[mir]

I don't think the speed you see is particularly slow. This is mine from a SSD direct attached to the motherboard (SATA 3)

CPU BOGOMIPS:                   22398.84
REGEX/SECOND:                  880593
HD SIZE:                             29.40 GB (/dev/mapper/pve-data)
BUFFERED READS:               242.24 MB/sec
AVERAGE SEEK TIME:           0.18 ms
FSYNCS/SECOND:                3462.96
DNS EXT:                            44.45 ms

 

[informant]

Do you mean, i must use a embedded RAID / or SATA connector directly without RAID and Proxmox install as ZFS in Raid 10?

regards

 

[mir]

If you want ZFS you should not be using a RAID controller what so ever. Either use a HBA or the SATA connectors on your motherboard.

 

[informant]

Ok, but if i dont have sata connectors on mainboard, i must use a raid controller - right. I use a hp dl360 g5 for this test.

regards

 

[mir]

Not a raid controller, a HBA. If a HBA is not available you will need to configure your RAID controller in JBOD mode. Or better flash your controller to TI mode.

 

[informant]

hi, do you mean this: http://www8.hp.com/h20195/v2/GetHTML.aspx?docname=c04150859

what you mean with "Or better flash your controller to TI mode." ?

regards

 

[Phinitris]

Hello,
you are wrong. You are getting your maximum speed.

Gbit/s != GB/s

1.5 Gbit/s = 1500 Mbit/s
1500 Mbit/s = 187.5 MB/s

So every disk can write theoretically 187.5 MB/s maximum. Calculated to your let's say "shitty" RAID 10 controller with BBU
266 MB/s is an approximate good result.

Best regards
Phinitris

 

[rene]

A host bus adapter from HP is the H222 where you could connect 4 internal and 4 external 6Gb/s SAS and SATA devices.

http://store.hp.com/webapp/wcs/stores/servlet/us/en/pdp/hp-h222-host-bus-adapter

 

[mir]

Sorry, it should have read IT mode (pass through mode, aka no raid functionality and no bios)

Your link points to a bad solution which requires the individual disks to be configured as individual raid0.

I my experience all HP raid controllers are crap. Go for LSI based controllers if you want to use ZFS. LSI based controllers can easily be flashed in IT mode.

 

[rene]

I have the HP H222 and it is a pure HBA. No raid at all.

 

[mir]

True, but the price is high for only 4 internal ports.

 

[rene]

It depends, I got mine on ebay for only 50 Euro.

 

[mir]

For the same price you would be able to get an IBM ServeRAID M1015 (LSI 1068) or M1115 (LSI 2008) on ebay.
Dell SAS 6/iR (LSI 1068) can also be found very cheap on ebay.

 

[informant]

hi all,

and which of the controllers is compatible with hp dl360 g5 server and have 3 or 6 Gbps?

regards