LVM storage + stripes parameter

FinnTux

Renowned Member
Aug 4, 2011
54
0
71
I'd like to bring up an old issue regarding lvm striping. It's been discussed about in this thread.
http://forum.proxmox.com/threads/6981-PVE-LVM-striping-on-iSCSI-storage-(with-multipath)

I've been experimenting with drbd a bit and got interesting results. I have two servers with two 1TB SATA disks each and dual 1 Gbit/s NICs bonded together with 802.3ad. Both disks are setup as their own drbd device and traffic is going in separate vlans. Volume group is created over these drbd devices. If VM disk is created manually with '--stripes 2' parameter I can see that drbd traffic is nicely balanced over different NICs and transfer speeds are pretty nice. This is obviously not the case without the parameter.

Host is Intel Q6600 with 8GB RAM
VM has 2 vcores and 1 GB of vRAM with minimal debian 7 installed.
DRBD 8.4.3
Test are made with bonnie

Default (no cache) VM disk. Large 16 GB test size
Code:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
changeme        16G   544  98 98631  18 74557  14  2964  97 247750  26 237.9   7
Latency             16467us     741ms     725ms   14770us   25376us     972ms
Version  1.96       ------Sequential Create------ --------Random Create--------
changeme            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 21926  37 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               441us     534us     905us     503us      46us     971us
1.96,1.96,changeme,1,1374168637,16G,,544,98,98631,18,74557,14,2964,97,247750,26,237.9,7,16,,,,,21926,37,+++++,+++,+++++,+++,+++++,+++,++
+++,+++,+++++,+++,16467us,741ms,725ms,14770us,25376us,972ms,441us,534us,905us,503us,46us,971us

Smaller test without bonnie -s parameter
Code:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
changeme         2G   549  98 166223  55 102893  20  2782  96 247848  24 441.5  10
Latency             28789us     129ms     145ms   18424us   18763us   71716us
Version  1.96       ------Sequential Create------ --------Random Create--------
changeme            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 32242  56 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               458us     560us     819us     432us      36us    1168us
1.96,1.96,changeme,1,1374174958,2G,,549,98,166223,55,102893,20,2782,96,247848,24,441.5,10,16,,,,,32242,56,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,28789us,129ms,145ms,18424us,18763us,71716us,458us,560us,819us,432us,36us,1168us


Same tests with writeback cache

16 GB
Code:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
changeme        16G   520  97 90191  20 62590  15  2980  97 246269  30 349.1  11
Latency             29155us     427ms     477ms    8851us   37501us    3060ms
Version  1.96       ------Sequential Create------ --------Random Create--------
changeme            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 25593  44 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               428us     589us     895us     473us      33us     986us
1.96,1.96,changeme,1,1374173786,16G,,520,97,90191,20,62590,15,2980,97,246269,30,349.1,11,16,,,,,25593,44,+++++,+++,+++++,+++,+++++,+++,+
++++,+++,+++++,+++,29155us,427ms,477ms,8851us,37501us,3060ms,428us,589us,895us,473us,33us,986us

without -s. Doesn't make that much sense because of the host cache but still the writes are replicated to the other DRBD host.

Code:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
changeme         2G   550  98 129077  50 133918  69  3002  97 856944  72  3651  71
Latency             27009us     250ms     335ms    2911us    7753us   14925us
Version  1.96       ------Sequential Create------ --------Random Create--------
changeme            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 31272  55 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency             16458us     555us     935us     470us      34us     910us
1.96,1.96,changeme,1,1374171575,2G,,550,98,129077,50,133918,69,3002,97,856944,72,3651,71,16,,,,,31272,55,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,27009us,250ms,335ms,2911us,7753us,14925us,16458us,555us,935us,470us,34us,910us


Althought thats not superfast enterprise stuff it is still replicated/shared storage with slow SATA disks. So would it be possible to have stripes and stripesize parameters? Not hard coded but simple input box in the LVM group storage creation form.

Thanks,
FinnTux
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!