MDADM / mdraid performance problems?

[check-ict]

Hello,

I have a small test setup of 3 proxmox servers based on RAID10 local storage.

Server1: 2.66GHz QC3330 xeon, 8GB mem, 4x 1TB sata, AHCI
Server2: 1.66GHz atom, 1GB mem, 4x 2TB sata, AHCI
Server3: 3.2GHz QC AMD PhenomII, 12GB mem, 4x 500GB sata, AHCI

All the servers have RAID10 setup with MDADM:
mdadm --create /dev/md0 -l 10 -c 1024 -p f2 -n 4 /dev/sd[abcd]1

After full sync I try to benchmark the md0 set, without a filesystem:
/dev/md0: Timing buffered disk reads: 254 MB in 3.00 seconds = 84.54 MB/sec

After the filesystem (ext3 or ext4) it's still under 100MB/sec. A single disk performs about 120MB/sec, so the RAID10 is even slower!

Changing readahead gives a better score in hdparm, but it doesn't really improve anything. What am I doing wrong?

I know you prefer hardware RAID, but I also see nice performance from people who use linux raid.

 

[jschellhaass]

First off don't use or consider hdparm as a benchmark tool.

Second, you provide very few details about your hardware setup.

What type of drives, make, model rotational speed? What type of controller/chipset are the drives attached too?

Third, what are your expectations with an md device.

Are there any errors in dmesg or /var/log/messages.

jeff

 

[check-ict]

The drives are Samsung Spinpoint F3 series. 500GB and 1000GB are 7200RPM SATA disks with a 120MB/sec read for each stand-alone disk. The 2000GB is 5400RPM but also gets around 120MB/sec for each disk.

Controllers are:
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 02)
And a cheap one:
11:00.0 RAID bus controller: Promise Technology, Inc. PDC40719 [FastTrak TX4300/TX4310] (rev 02)

Bonnie++:
Version 1.03d
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 46502 66 48604 11 26203 3 75442 87 99888 6 293.5 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
hostname,16G,46502,66,48604,11,26203,3,75442,87,99 888,6,293.5,0,16,+++++,+++,+++++,+++,+++++,+++,+++ ++,+++,+++++,+++,+++++,+++


I expect the RAID to be faster then a stand-alone disk.

 

[jschellhaass]

How is the Intel ich9r set in the bios? It should not be set for raid. It should be set for achi.

jeff

 

[check-ict]

As mentioned in the first post, it's set to AHCI

 

[jschellhaass]

This link may help you optimize the file system considering your chunk size.

http://busybox.net/~aldot/mkfs_stride.html

jeff