Very high IO for SAS2 RAID10

Andre Köhler

New Member
Jan 9, 2012
16
1
1
Hattingen, Germany
Hi all,
i have all update installed (i think so) an now i've made a convert from vmdk to raw with qemu-img.

And what I see?

Read/Write only with 30 MByte/s

I have 4 SAS2 HDD in RAID10 an Convert from /var/lib/vz/source.vmdk to /var/lib/vz/target.raw

any Idea why it is soooo slowly?

The Raid Status is ok

Read an Write Cache is ON
No Rebuild or verifikation
256k Stripe

it is a 3Ware


/c2/u0 status = OK
/c2/u0 is not rebuilding, its current state is OK
/c2/u0 is not verifying, its current state is OK
/c2/u0 is initialized.
/c2/u0 Write Cache = on
/c2/u0 Read Cache = Intelligent
/c2/u0 volume(s) = 1
/c2/u0 name = InfoTipRAID10
/c2/u0 serial number = A00YFM131943610055F2
/c2/u0 Ignore ECC policy = off
/c2/u0 Auto Verify Policy = on
/c2/u0 Storsave Policy = balance
/c2/u0 Command Queuing Policy = on
/c2/u0 Rapid RAID Recovery setting = all
Unit UnitType Status %RCmpl %V/I/M VPort Stripe Size(GB)
------------------------------------------------------------------------
u0 RAID-10 OK - - - 256K 3725.27
u0-0 RAID-1 OK - - - - -
u0-0-0 DISK OK - - p11 - 1862.63
u0-0-1 DISK OK - - p10 - 1862.63
u0-1 RAID-1 OK - - - - -
u0-1-0 DISK OK - - p9 - 1862.63
u0-1-1 DISK OK - - p8 - 1862.63
u0/v0 Volume - - - - - 3725.27





I hope anyone can help me...


Thanks
 
i cant say that, because i began with this version.

But in my optionion... SAS2 HDD... with 6Gbit/s .... read/write as the same time... there is 30MB/s not just a "little" do slow or?

But i don't know where i should search the mistacke...
 
i've tested this on my fileserver, but remember i have only a software raid10...

first test -> write a new 2GB file

[root@mars data]# dd if=/dev/zero of=test-file bs=1M count=2000 oflag=direct
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 16.8462 seconds, 124 MB/s


second test -> rewrite a file where the source and target are in the same location:

[root@mars data]# dd if=test-file of=test-file2 bs=1M count=2000 oflag=direct
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 54.4193 seconds, 38.5 MB/s
 
yeah look at that

that is software raid and faster than my hardware raid... why that...

this evening/night iam going to start with knopix cd and then i test it with hdparm... if it the same "sh**" ... i have a hardware defekt...
 
i don't think that my software raid is faster then your's

what's your overall performance of this raid?
can you post us the result of a write test?

show us:
dd if=/dev/zero of=/var/lib/vz/test-file bs=1M count=2000 oflag=direct
 
root@proxmox1:~# dd if=/dev/zero of=/var/lib/vz/test-file bs=1M count=2000 oflag=direct
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 13.3044 s, 158 MB/s

faster... mmh...

so what ive checked:

yesterday evening at 11 pm hdparm /dev/sda = ~ 300 MB/s

so... maybe...

if the VM have load... the normal system goes down?

that can be the point...

at 11 pm the SBS 2003 didn't had any load...

now SBS is on LOAD (20 Users connected)

and now hdparm:

root@proxmox1:~# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 23764 MB in 2.00 seconds = 11894.23 MB/sec
Timing buffered disk reads: 324 MB in 3.00 seconds = 107.86 MB/sec


faster then yesterday... but slowly for SAS2
 
root@proxmox1:~# dd if=/dev/zero of=/var/lib/vz/test-file bs=1M count=2000 oflag=direct
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 13.3044 s, 158 MB/s

faster... mmh...

so what ive checked:

yesterday evening at 11 pm hdparm /dev/sda = ~ 300 MB/s

so... maybe...

if the VM have load... the normal system goes down?

that can be the point...

at 11 pm the SBS 2003 didn't had any load...

now SBS is on LOAD (20 Users connected)

and now hdparm:

root@proxmox1:~# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 23764 MB in 2.00 seconds = 11894.23 MB/sec
Timing buffered disk reads: 324 MB in 3.00 seconds = 107.86 MB/sec


faster then yesterday... but slowly for SAS2
Hi,
if other processes making IO can you see with
Code:
iostat -dm 5 sda   # or sdb ...
You must install sysstat to have iostat

Udo
 
root@proxmox1:/# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 21754 MB in 2.00 seconds = 10887.72 MB/sec
Timing buffered disk reads: 626 MB in 3.00 seconds = 208.58 MB/sec
root@proxmox1:/# iostat -dm 5 sda
Linux 2.6.32-6-pve (proxmox1) 02/02/2012 _x86_64_ (8 CPU)
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 1407.62 3.52 7.71 724581 1587223
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 16.20 0.03 0.13 0 0
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 15.20 0.01 0.08 0 0
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 9.60 0.01 0.05 0 0
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 13.00 0.03 0.07 0 0





now it's "fast" with 200MB and no other is rewrite... mmh but why sometimes so slowly...
 
just my 2 cents, hdparm is probably not the right tool for reliable IO benchmarks ...
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!