bad performace of cyrus server in kvm-guest

[grl]

Hi!

Tried to setup a proxmox-cluster for some heavy testing. Works like charm so far. Now I moved a cyrus/postfix/apache/mysql server from real hardware in a kvm guest. The performance of the apache/mysql-combo seems to be good, but cyrus seems to be really slow. Several users report Outlook going offline and being unable to send mails. When trying to restart cyrus on the server I get lot's of "imapd <defunct>" processes on my server, and a restart of cyrus fails with "shutdown failed".
I can't even SIGKILL that processes and have to restart the whole server.

The exactly same installation worked perfect for quite some time on "real" hardware - that's why i choose this machine for testing. I just copied the files to a new kvm guest, fixed grub and started up.

My setup is:
two Xeon E3-1260L with 16GB Ram each, a SSD as Bootdist and a 2TB SATA-Drive used with DRBD as shared storage. I used not the iso-installer but a native Debian squeeze. DRBD and Clustering is done by a dedicated 1GBit NIC, another NIC is connected as VMBR0 to the provider ownded switch using 10BaseT half duplex.

The physical hosts have an identic Setup:

pve-manager: 2.0-38 (pve-manager/2.0/af81df02)
running kernel: 2.6.32-7-pve
proxmox-ve-2.6.32: 2.0-60
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-23
qemu-server: 2.0-25
pve-firmware: 1.0-15
libpve-common-perl: 1.0-17
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-12
vncterm: 1.0-2
vzctl: 3.0.30-2pve1
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-5
ksm-control-daemon: 1.1-1

The virtual host is defined as follows:

boot: c
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
keyboard: de
memory: 2048
name: webtest
net0: virtio=52:B4:B3:DD:72:80,bridge=vmbr0
onboot: 1
ostype: l26
sockets: 2
virtio0: LVM01:vm-104-disk-1,cache=writethrough

The old physical machine was a P4, single core with 2GB Memory

As exactly that setup worked perfectly so far, I think the cyrus/postfix/apache config is ok.

Anyone a good idea how to proceed?

regards
Lukas

 

[tom]

you do not have a raid controller with BBU for your DRBD? I assume this can be an issue (you use internal metadata?). move the VM to a another (non-drbd) disk to eliminate any DRBD related issue.

second, give all details about your VM - which OS?

third, do not use cache=writethrough, use cache=none

 

[grl]

you do not have a raid controller with BBU for your DRBD? I assume this can be an issue (you use internal metadata?).

No, I don't use a raid controller - as I use just one single disk. And yes, internal metadata. I know about perfomance problems in this setup but did'nt expect them to be that dramatic.

For the final setup I'm not shure how to do: Building a raid with raid-controller and BBU is not an option as there's just space for two disks in the server-housing (one Boot-SSD, one Storage-HDD).
Using a raid-controller with BBU for just one disk: I'm not shure if that works.
What about using a SSD? Any experiences on that?

move the VM to a another (non-drbd) disk to eliminate any DRBD related issue.

This is a bit of a problem at the moment, as there ist not enought non-DRBD space in the machine - so that's not what can be done quite easyly. I would prefer not to change the HW-Setup before having decided on RAID-Controller or SATA.

second, give all details about your VM - which OS?

Sorry, forgot that...
Debian lenny, Kernel 2.6.32, cyrus 2.2.13, postfix 2.7.1, apache 2.2.9, mysql 5.1.51

third, do not use cache=writethrough, use cache=none

done, another test running.
Can you point me to some documentation about that parameter? I didn't find much on that...

regards
Lukas

 

[tom]

No, I don't use a raid controller - as I use just one single disk. And yes, internal metadata. I know about perfomance problems in this setup but did'nt expect them to be that dramatic.

For the final setup I'm not shure how to do: Building a raid with raid-controller and BBU is not an option as there's just space for two disks in the server-housing (one Boot-SSD, one Storage-HDD).
Using a raid-controller with BBU for just one disk: I'm not shure if that works.
What about using a SSD? Any experiences on that?

see also Florian blog, http://fghaas.wordpress.com/2009/08/20/internal-metadata-and-why-we-recommend-it/
regarding DRBD tuning and general questions, I recommend to post in drbd mailing list or even better, go for a support contract with Linbit to find the best setup.

This is a bit of a problem at the moment, as there ist not enought non-DRBD space in the machine - so that's not what can be done quite easyly. I would prefer not to change the HW-Setup before having decided on RAID-Controller or SATA.


Sorry, forgot that...
Debian lenny, Kernel 2.6.32, cyrus 2.2.13, postfix 2.7.1, apache 2.2.9, mysql 5.1.51

seems fine.

done, another test running.
Can you point me to some documentation about that parameter? I didn't find much on that...

regards
Lukas

just see 'man kvm'.