OpenVZ containers very slow performance after upgrade to 1.5

[llazzaro]

I update to 1.5, but my the host is logging a lo of I/O delays since upgrade.

Any help to debug my problem?
with 1.3 I never had problems.

Its really slow, like "ls" command takes 3 seconds. I dont know what that I/O comes from!

 

[dietmar]

What is the output of:

# pveversion -v

You already rebooted the host?

 

[llazzaro]

I already reboot,of course after upgrade.
Today I will reboot it again.

this is the output.

pve-manager: 1.5-7 (pve-manager/1.5/4660)
running kernel: 2.6.24-8-pve
pve-kernel-2.6.24-8-pve: 2.6.24-16
pve-kernel-2.6.24-7-pve: 2.6.24-11
qemu-server: 1.1-11
pve-firmware: 1.0-3
libpve-storage-perl: 1.0-10
vncterm: 0.9-2
vzctl: 3.0.23-1pve8
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1

There is something new near the upgrade. I am running a container as gateway, with shorewall.

 

[tom]

you didn´t follow the installation/update guide, the virtual package 'proxmox-ve-2.6.24' is missing, therefore you also have an old kernel.

to fix:

aptitude install proxmox-ve-2.6.24

see also http://pve.proxmox.com/wiki/Downloads

 

[llazzaro]

you didn´t follow the installation/update guide, the virtual package 'proxmox-ve-2.6.24' is missing, therefore you also have an old kernel.

to fix:

aptitude install proxmox-ve-2.6.24

see also http://pve.proxmox.com/wiki/Downloads

Ok, I follow that link and install the proxmox-ve-2.6.24. I was using selfmade kernel (from proxmox of course).

IFor now it seems all ok, I need to wait a few hours to see if everything is OK.

I attached the I/O stats from proxmox hardware node, it was impossible to write via ssh (like 10 seconds for each char I write!)

seems ok now.

 

[llazzaro]

I still with issues, this is the new output of pveversion -v

pve-manager: 1.5-7 (pve-manager/1.5/4660)
running kernel: 2.6.24-10-pve
proxmox-ve-2.6.24: 1.5-21
pve-kernel-2.6.24-8-pve: 2.6.24-16
pve-kernel-2.6.24-7-pve: 2.6.24-11
pve-kernel-2.6.24-10-pve: 2.6.24-21
qemu-server: 1.1-11
pve-firmware: 1.0-3
libpve-storage-perl: 1.0-10
vncterm: 0.9-2
vzctl: 3.0.23-1pve8
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.11.1-2

The problem seems to appear after a few hours, and there are a lot of harddisk activity.
Check if any container was using a lot of ram, and no problem with this.
I deactivate a lot of container, and only leave a few ones up.
Also with new kernel, softirq was taking %100 of one CPU, with old kernel never happen this.
softirq problem seems to be random, could be an attack from outside this thing?

thanks!

 

[llazzaro]

Well the problem persists, If I have some time tomorrow I will try to swith to other kernel. I Have a container with MySQL, stopping the service reduces I/O to 10% (but I/O problem stills there reduced but there).
At container, today I made an upgrade of soft (mysql version,etc with apt-get) nothing got fixed.

So chaging the kernel version dont solve the problem. I am goind to 2.6.18, hope it gets fixed.

I never got so much I/O utilization never, until I upgrade to 1.5. I am sure it could be a configuration issue. Hope my harddrive will be not dead by next week when I have some time (hardrive activiy its very high).

 

[llazzaro]

I fixed the problem switching to 2.6.18-2-pve, when I have some time I will try to find the problem sin ce 2.6.24 was working at 1.3 with no problems.
This only happens when using openvz containers