MDADM / mdraid performance problems?

check-ict

Well-Known Member
Apr 19, 2011
102
18
58
Hello,

I have a small test setup of 3 proxmox servers based on RAID10 local storage.

Server1: 2.66GHz QC3330 xeon, 8GB mem, 4x 1TB sata, AHCI
Server2: 1.66GHz atom, 1GB mem, 4x 2TB sata, AHCI
Server3: 3.2GHz QC AMD PhenomII, 12GB mem, 4x 500GB sata, AHCI

All the servers have RAID10 setup with MDADM:
mdadm --create /dev/md0 -l 10 -c 1024 -p f2 -n 4 /dev/sd[abcd]1

After full sync I try to benchmark the md0 set, without a filesystem:
/dev/md0: Timing buffered disk reads: 254 MB in 3.00 seconds = 84.54 MB/sec

After the filesystem (ext3 or ext4) it's still under 100MB/sec. A single disk performs about 120MB/sec, so the RAID10 is even slower!

Changing readahead gives a better score in hdparm, but it doesn't really improve anything. What am I doing wrong?

I know you prefer hardware RAID, but I also see nice performance from people who use linux raid.
 
First off don't use or consider hdparm as a benchmark tool.

Second, you provide very few details about your hardware setup.

What type of drives, make, model rotational speed? What type of controller/chipset are the drives attached too?

Third, what are your expectations with an md device.

Are there any errors in dmesg or /var/log/messages.

jeff
 
The drives are Samsung Spinpoint F3 series. 500GB and 1000GB are 7200RPM SATA disks with a 120MB/sec read for each stand-alone disk. The 2000GB is 5400RPM but also gets around 120MB/sec for each disk.

Controllers are:
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 02)
And a cheap one:
11:00.0 RAID bus controller: Promise Technology, Inc. PDC40719 [FastTrak TX4300/TX4310] (rev 02)

Bonnie++:
Version 1.03d
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 46502 66 48604 11 26203 3 75442 87 99888 6 293.5 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
hostname,16G,46502,66,48604,11,26203,3,75442,87,99 888,6,293.5,0,16,+++++,+++,+++++,+++,+++++,+++,+++ ++,+++,+++++,+++,+++++,+++


I expect the RAID to be faster then a stand-alone disk.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!