Sluggish Performance

tylerdurden81

Member
Jan 8, 2011
124
0
16
I have moved many VM's from earlier versions of proxmox to proxmox 2.0 I have noticed that the performance has gone down severely. I initially thought it was the local storage but have noticed the VM's that are on a san (as they were in proxmox 1.9) have the same performance issues.

Here is output of pveperf

pveperf
CPU BOGOMIPS: 230368.98
REGEX/SECOND: 745583
HD SIZE: 94.49 GB (/dev/mapper/pve-root)
BUFFERED READS: 336.98 MB/sec
AVERAGE SEEK TIME: 21.13 ms
FSYNCS/SECOND: 28.45
DNS EXT: 72.28 ms
DNS INT: 133.99 ms

I have seen other posts where FSYNCS should be higher. Is there a way to fix that?


Here is a pveperf on one of the san systems. I notice that the fsyncs are higher .. i am not sure i did the command 100% correct.

pveperf /dev/san5
CPU BOGOMIPS: 230368.98
REGEX/SECOND: 716351
HD SIZE: 43.26 GB (udev)
FSYNCS/SECOND: 39415.02
DNS EXT: 96.40 ms
DNS INT: 63.55 ms

Even though they are higher the VM performance seems to be the same. For example it used to take about 9minutes to install windows 2008 in proxmox 1.9. Now it takes about 50 minutes. This dosn't matter what storage I use.

pveperf information might be skewed because there are close to 100 VM's running on the system right now.
 
I have moved many VM's from earlier versions of proxmox to proxmox 2.0 I have noticed that the performance has gone down severely. I initially thought it was the local storage but have noticed the VM's that are on a san (as they were in proxmox 1.9) have the same performance issues.

Here is output of pveperf

pveperf
CPU BOGOMIPS: 230368.98
REGEX/SECOND: 745583
HD SIZE: 94.49 GB (/dev/mapper/pve-root)
BUFFERED READS: 336.98 MB/sec
AVERAGE SEEK TIME: 21.13 ms
FSYNCS/SECOND: 28.45
DNS EXT: 72.28 ms
DNS INT: 133.99 ms

I have seen other posts where FSYNCS should be higher. Is there a way to fix that?


Here is a pveperf on one of the san systems. I notice that the fsyncs are higher .. i am not sure i did the command 100% correct.

pveperf /dev/san5
CPU BOGOMIPS: 230368.98
REGEX/SECOND: 716351
HD SIZE: 43.26 GB (udev)
FSYNCS/SECOND: 39415.02
DNS EXT: 96.40 ms
DNS INT: 63.55 ms

Even though they are higher the VM performance seems to be the same. For example it used to take about 9minutes to install windows 2008 in proxmox 1.9. Now it takes about 50 minutes. This dosn't matter what storage I use.

pveperf information might be skewed because there are close to 100 VM's running on the system right now.
Hi,
you need an mounted filesystem for pveperf. To test the (read) speed of san5, you can create an logical volum on the vg, create an filesystem (ext3), mount and do an pveperf on this mountpoint.

But "measuring" with 100 running VM makes no sense. On a calm system you can look for fsyncs/s and speed and if you can't reach the speed in production, you must find the processes which "eat" the performance.

About win-installaton. What load shows top on the host? (and wait)?

Udo
 
I am not sure what you mean here.
Hi,
if you install windows in a vm and run top on the host, there is an output like this:
Code:
top - 19:05:23 up 2 days, 20:35,  1 user,  [B]load average: 0.30, 0.10, 0.03[/B]
Tasks: 227 total,   1 running, 226 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.6%us,  4.4%sy,  0.0%ni, 78.9%id, [B]11.9%wa[/B],  0.0%hi,  0.1%si,  0.0%st
Mem:  16171292k total,  5350828k used, 10820464k free,   134804k buffers
Swap:  8388600k total,        0k used,  8388600k free,  1060836k cached
here the windowsfiles was expanded (9%) - IO and cpu.

Just do an installation - take less than 10 minutes on pve2.
Do you use more than 1cpu/core? Does the same happens with only 1 cpu/core?

Udo
 
Ok.. that is what i was thinking you ment. All of the VM's I have been testing have been using one core. I will do a install and take a look and post the results.
 
This is windows 2008 at around %57 expanding its been running for about a hour.

This is the VM ....... 225242 root 20 0 2257m 2.0g 1980 S 1 2.3 58:23.73 kvm


This is the rest of the output

top
top - 14:54:16 up 3 days, 2:33, 1 user, load average: 0.54, 1.11, 1.20
Tasks: 942 total, 3 running, 939 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.1%us, 0.9%sy, 0.0%ni, 96.7%id, 1.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 90735832k total, 24952688k used, 65783144k free, 61912k buffers
Swap: 90177528k total, 0k used, 90177528k free, 2435796k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
139694 root 20 0 726m 534m 2988 R 18 0.6 571:06.89 kvm
181464 root 20 0 599m 412m 2956 S 10 0.5 319:01.72 kvm
36815 root 20 0 2227m 1.7g 2992 S 9 2.0 655:16.45 kvm
643104 root 20 0 1170m 1.0g 2036 S 7 1.2 13:45.24 kvm
662047 root 20 0 2200m 2.0g 2952 S 7 2.3 3:21.32 kvm
86448 root 20 0 1241m 1.0g 2968 S 6 1.2 2136:18 kvm
644613 root 20 0 3246m 3.0g 2984 S 6 3.5 59:42.78 kvm
42579 root 20 0 728m 537m 2972 S 6 0.6 236:39.50 kvm
17014 root 20 0 1174m 1.0g 2992 S 4 1.2 320:55.65 kvm
45354 root 20 0 926m 791m 2968 S 4 0.9 360:29.63 kvm
73140 root 20 0 658m 531m 2040 S 4 0.6 262:30.29 kvm
674545 www-data 20 0 277m 41m 4284 S 2 0.0 0:00.32 apache2
674449 www-data 20 0 277m 41m 4376 S 2 0.0 0:00.69 apache2
3050 root 20 0 167m 28m 3964 S 1 0.0 26:29.74 pvestatd
10660 root 20 0 659m 519m 2948 S 1 0.6 79:30.76 kvm
60095 root 20 0 660m 533m 2036 S 1 0.6 90:15.87 kvm
673520 root 20 0 193m 39m 5512 S 1 0.0 0:03.13 pvedaemon
674487 root 20 0 19744 2080 1016 R 1 0.0 0:00.37 top
56560 root 20 0 660m 139m 2976 S 1 0.2 57:14.12 kvm
61722 root 20 0 2200m 2.0g 2984 S 1 2.3 79:20.31 kvm
225242 root 20 0 2257m 2.0g 1980 S 1 2.3 58:23.73 kvm
263116 root 20 0 1168m 689m 2052 R 1 0.8 109:09.39 kvm
324633 root 20 0 655m 533m 1996 S 1 0.6 144:38.12 kvm
671188 root 20 0 193m 39m 5760 S 1 0.0 0:10.15 pvedaemon
673541 root 20 0 193m 39m 5512 S 1 0.0 0:03.17 pvedaemon
57 root 20 0 0 0 0 S 0 0.0 1:23.80 ksoftirqd/13
113 root 20 0 0 0 0 S 0 0.0 0:42.42 ksoftirqd/27
2197 root 20 0 419m 57m 31m S 0 0.1 8:23.53 pmxcfs
2479 root 0 -20 0 0 0 S 0 0.0 3:51.29 iscsi_q_9
12837 root 20 0 677m 540m 2972 S 0 0.6 312:23.15 kvm
57830 root 20 0 662m 539m 2984 S 0 0.6 248:17.13 kvm
662060 root 20 0 0 0 0 S 0 0.0 0:00.38 kvm-pit-wq
1 root 20 0 8360 808 676 S 0 0.0 0:05.18 init
2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0 0.0 0:04.33 migration/0
4 root 20 0 0 0 0 S 0 0.0 0:00.81 ksoftirqd/0
5 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
6 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/0
7 root RT 0 0 0 0 S 0 0.0 0:03.82 migration/1
8 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
9 root 20 0 0 0 0 S 0 0.0 0:11.53 ksoftirqd/1
10 root RT 0 0 0 0 S 0 0.0 0:02.52 watchdog/1
11 root RT 0 0 0 0 S 0 0.0 0:03.93 migration/2
12 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/2
13 root 20 0 0 0 0 S 0 0.0 0:38.06 ksoftirqd/2
14 root RT 0 0 0 0 S 0 0.0 0:02.83 watchdog/2
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!