Measure PVE performance

N

nick

Renowned Member
Mar 28, 2007
364
1
83
Hi All,

I intend to create a tutorial and explain how to create a correct PVE server and for this I need to know how to measure PVE parameters; what are the benchmark commands?

If is possible to receive some values (let say "normal working" values) to try to see where we need to concentrate.

For example: what are the values obtained with a RAID controller in RAID 1/0/10 and what are the value with a standard SATA disk...to compare performace value.
 
nick said:
Hi All,

I intend to create a tutorial and explain how to create a correct PVE server and for this I need to know how to measure PVE parameters; what are the benchmark commands?

If is possible to receive some values (let say "normal working" values) to try to see where we need to concentrate.

For example: what are the values obtained with a RAID controller in RAID 1/0/10 and what are the value with a standard SATA disk...to compare performace value.
Click to expand...

Hi,
i can give you a good value (i think it's a good value) for a raid-10 with 4 SAS-drives (a 450GB) and a areca-raid controller:
Code: 
pveperf /var/lib/vz
CPU BOGOMIPS:      27293.47
REGEX/SECOND:      1101025
HD SIZE:           543.34 GB (/dev/mapper/pve-data)
BUFFERED READS:    497.18 MB/sec
AVERAGE SEEK TIME: 5.51 ms
FSYNCS/SECOND:     5646.17
DNS EXT:           100.62 ms
DNS INT:           1.90 ms
sometimes i got "only" 430MB/s

A raid-1 (system + backup) with two 2T-SATA on the same controller:
Code: 
BUFFERED READS:    110.64 MB/sec
AVERAGE SEEK TIME: 7.66 ms
FSYNCS/SECOND:     3575.26

Udo
 
nick said:
Hi All,

I intend to create a tutorial and explain how to create a correct PVE server and for this I need to know how to measure PVE parameters; what are the benchmark commands?

If is possible to receive some values (let say "normal working" values) to try to see where we need to concentrate.

For example: what are the values obtained with a RAID controller in RAID 1/0/10 and what are the value with a standard SATA disk...to compare performace value.
Click to expand...

Hi,

i can give you the performance Data of a HP DL360 G6 with a Smart Array P410i 256MB BBWC RAID Controller.

Raid Level 10

The HDDs are 4x HP 2,5i 146GB 10Krpm SAS 6Gb/s DP

CPU BOGOMIPS: 16000.12
REGEX/SECOND: 583739
HD SIZE: 67.18 GB (/dev/pve/root)
BUFFERED READS: 238.39 MB/sec
AVERAGE SEEK TIME: 5.06 ms
FSYNCS/SECOND: 3805.86
 
I just compare some values:

1) RAID configuration: FSYNCS/SECOND: >3000
2) STANDARD S-ATA (7200 rpm disk): FSYNCS/SECOND: 100-200
3) SAS 15k (no raid): FSYNCS/SECOND: max 300.

So the disk performance is connected to the RAID CARD. Without this, the PVE performance is LOW.

Please post more resuts for "pveperf" command
 
Adaptec RAID 2405
RAID1
S-ATA
Code: 
CPU BOGOMIPS:      31921.11
REGEX/SECOND:      436561
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    164.61 MB/sec
AVERAGE SEEK TIME: 8.59 ms
FSYNCS/SECOND:     2145.57
 
Last edited:
ITN said:
Adaptec RAID 2405
RAID1
S-ATA
Code: 
CPU BOGOMIPS:      31921.11
REGEX/SECOND:      436561
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    164.61 MB/sec
AVERAGE SEEK TIME: 8.59 ms
FSYNCS/SECOND:     2145.57
Click to expand...

these must be the numbers with write cache enabled - and this controller has no BBU, so this configuration can lead to data loss in a power failure. good enough for a test machine, but risky.
 
tom said:
these must be the numbers with write cache enabled - and this controller has no BBU, so this configuration can lead to data loss in a power failure. good enough for a test machine, but risky.
Click to expand...

Right, i enabled it only for the test.

Without write cache:

Code: 
CPU BOGOMIPS:      31921.11
REGEX/SECOND:      430529
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    174.82 MB/sec
AVERAGE SEEK TIME: 9.63 ms
FSYNCS/SECOND:     692.81

Btw why is a UPS not enough?
 
Last edited:
As long as your 100% certain the UPS will shut down the machine before the power goes out, you *should* be fine.

However if the UPS dies/malfunctions and the system crashes hard, and there is data in the cache waiting to be written to the drives, you might have massive data corruption when the machine comes back up.

The BBU ensures anything is cache is written to the drives so even in the event of a hard power failure you know your data should be safe and sound.

BBU is -always- recommended if your data is worth anything at all..
 
ITN said:
Right, i enabled it only for the test.

Without write cache:

Code: 
CPU BOGOMIPS:      31921.11
REGEX/SECOND:      430529
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    174.82 MB/sec
AVERAGE SEEK TIME: 9.63 ms
FSYNCS/SECOND:     692.81
Click to expand...

and this is with hard drive cache enabled. using the ASM you can turn HDD cache on or off. Only off is really secure.

ITN said:
Btw why is a UPS not enough?
Click to expand...

not always, e.g. when your power supply or mainboard is dying, a UPS does not help.
 
resignation said:
As long as your 100% certain the UPS will shut down the machine before the power goes out, you *should* be fine.

However if the UPS dies/malfunctions and the system crashes hard, and there is data in the cache waiting to be written to the drives, you might have massive data corruption when the machine comes back up.

The BBU ensures anything is cache is written to the drives so even in the event of a hard power failure you know your data should be safe and sound.

BBU is -always- recommended if your data is worth anything at all..
Click to expand...

Thanks, i see its a question of probability. ;)
 
I test more then 10 configurations (3 servers) and I realize that:

If someone want to configure a low cost server with normal configuration (used with maximum 5 VM) can use one 7200 rpm server with 64MB cache. I obtain on this test FSYNCS/SECOND >1000 even when the VM was started. In this case all the time have another disk dedicated for backup.

I compare RAID 5 with RAID 0 and RAID10 on the same controller with 256 MB cache: the best perfomance was obtained on RAID 0. Second was RAID 10 and last RAID 5 : FSYNCS/SECOND > 2500

Personal, after all the test I can say that into a small bussines can be used a Low cost server (1 SBS Server + 1 PMG + 1 Linux Server for web sevices). In this case we reduce the costs.

If you bussiness have mothe then 15 employers, configurate the PVE server with a RAID controller with minimum 256 MB Cache. Otherwise, the performance on exchange and Offline files functions decrese. The perfomance for PMG and Linux are not affected so much (there are OpenVZ machines)

These are my fisrt conclusions. I will try to prepare a document with tests results.
 
I have measured the performance of two new HP Servers with Proxmox 1.6.

HP ML350 G6 with a Smart Array P410i 256MB BBWC RAID Controller and 2 x 300 GB 15krpm SAS HDDs.

Raid 1:

CPU BOGOMIPS: 17060.36
REGEX/SECOND: 740076
BUFFERED READS: 129.33 MB/sec
AVERAGE SEEK TIME: 4.06 ms
FSYNCS/SECOND: 2815.86 ms

Raid 0:

CPU BOGOMIPS: 17060.91
REGEX/SECOND: 731750
BUFFERED READS: 366.06 MB/sec
AVERAGE SEEK TIME: 4.00 ms
FSYNCS/SECOND: 2755.58 ms


And a ML110 G6 with a WD Raptor 10k rpm SATA HDD at the onboard Controller.

CPU BOGOMIPS: 19151.25
REGEX/SECOND: 860122
BUFFERED READS: 139.40 MB/sec
AVERAGE SEEK TIME: 4.94 ms
FSYNCS/SECOND: 1298.84 ms
 
Sorcerer said:
.....

And a ML110 G6 with a WD Raptor 10k rpm SATA HDD at the onboard Controller.

CPU BOGOMIPS: 19151.25
REGEX/SECOND: 860122
BUFFERED READS: 139.40 MB/sec
AVERAGE SEEK TIME: 4.94 ms
FSYNCS/SECOND: 1298.84 ms
Click to expand...

thanks for sharing your results - one question to the raptor: which model, please post the model number and size.
 
thanks for the info!
 
Hi,
I made just some test with sas-drives.

To remark the dependencies between number of disks in a raid-10:

raid-10 with 4 disks:
Code: 
CPU BOGOMIPS:      24082.40
REGEX/SECOND:      1021282
HD SIZE:           196.86 GB (/dev/mapper/test--sas)
BUFFERED READS:    [B]512.45 MB/sec[/B]
AVERAGE SEEK TIME: 4.08 ms
FSYNCS/SECOND:     5714.96

the same with 6 disks:
Code: 
CPU BOGOMIPS:      24082.39
REGEX/SECOND:      986071
HD SIZE:           196.86 GB (/dev/mapper/testvg-test)
BUFFERED READS:    [B]869.12 MB/sec[/B]
AVERAGE SEEK TIME: 3.84 ms
FSYNCS/SECOND:     5859.23
the same with 8 disks:
Code: 
CPU BOGOMIPS:      24080.90
REGEX/SECOND:      962568
HD SIZE:           196.86 GB (/dev/mapper/testvg-test)
BUFFERED READS:    [B]999.95 MB/sec[/B]
AVERAGE SEEK TIME: 3.75 ms
FSYNCS/SECOND:     5983.93
The difference from 4 to 6 drives are pleasant... (in the final i will use only 4 drives on each node).

The raidcontroller is an areca ARC-1680ix-24 with 512MB onboard RAM (will be ugraded). The sas-drives are hitachi ultrastar 15K600.

Udo
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back