[SOLVED] Slow Debian 4.0 guest after migration from physical hardware (CPU usage 100% in the dashboard)

P

Primoz Cankar

New Member
Feb 22, 2016
15
0
1
cankar.si
I've migrated an old Debian 4.0 box to proxmox using clonezilla. The guest has mdadm raid setup and so I've reconfigured it to work in the VM. When I run it on proxmox using vmdk disk image the performance is just unbearable. The Host system says that the cpu is running at 100% but when I go into the VM top command shows everything at 0. Although the load average is extremly high around 8. When I run the same VM using VirtualBox it runs without a hitch. The host machine is an i7 920with 6Gb of DDR3 and the guest is set to 512MB. This is the same for Virtualbox except that VirtualBox is running on a i5 2xxx.
If anyone has any idea please let me know.
 
IDE is connection type vmdk is disk image type. I'm using IDE as the connection type. If I use virtio or sata it won't work because then I have to reconfigure grub to work with one of those and perhaps virtio is not even possible since the guest is Debain 4.0.
 
I also migrated an old Debian etch box without any performance problems half a year ago. Booting is not fast, but that's due to the old linux which does not know anything about virtualization.

IIRC IDE was the only option for such an old linux system.
 
mir said:
You should not use vmdk. with QEMU you should use Qcow2 or raw.
Click to expand...
The first thing I've used was qcow2, then I tried raw and after both failed I converted to vmdk so that I could transfer to another hyper-visor to test it there. On all three of them I had the same problem and this all running on an i7 with raid 1 on zfs. When I install a fresh Ubuntu 14.04.4 machine on the same computer the VM works flawlessly.
 
LnxBil said:
I also migrated an old Debian etch box without any performance problems half a year ago. Booting is not fast, but that's due to the old linux which does not know anything about virtualization.

IIRC IDE was the only option for such an old linux system.
Click to expand...
I wouldn't mind if it took long to boot but when it reaches the login prompt and I log in it just so slow. And the graph on the control panel is showing disk speeds around 500kB/s.
 
More questions:
  • Maybe there are some strange modules loaded?
  • Does dmesg show anything interesting?
  • Is this a stock Etch kernel?
  • Is this a amd64 or i386 system? Maybe setting the virtual CPU to something older helps
  • What about /proc/mdstat? Your local disk is still partitioned as software raid metadata, isn't it?
  • What about analysing via qemu monitor? The strange background load has to come from somewhere.
 
  • Like
Reactions: Primoz Cankar
LnxBil said:
More questions:
  • Maybe there are some strange modules loaded? - I'll check this tomorrow, there shouldn't be
  • Does dmesg show anything interesting? - I'll check this tomorrow
  • Is this a stock Etch kernel? - It should be
  • Is this a amd64 or i386 system? Maybe setting the virtual CPU to something older helps - it's a i386 and I've set the kvm32 for the virtualizer
  • What about /proc/mdstat? Your local disk is still partitioned as software raid metadata, isn't it? - I'll check this tomorrow
  • What about analysing via qemu monitor? The strange background load has to come from somewhere. - I'll check this tomorrow
Click to expand...

The thing is I don't know much about this system since it was purposely built by some other company to host an app database.
It's still puzzling me though that VirtualBox runs it perfectly.

Anyway, thanks for the tips @LnxBil. I'll update the thread when I get some answers to this questions.
 
This is what I got from the guest and host system. I don't really see anything that would be of interest except that piix3 in dmesg on the guest.

# HOST

## dmesg -w
Code: 
[  74.375842] device tap100i0 entered promiscuous mode
[  74.380944] vmbr0: port 2(tap100i0) entered forwarding state
[  74.380955] vmbr0: port 2(tap100i0) entered forwarding state
[  77.368002] kvm: zapping shadow pages for mmio generation wraparound
[  77.368666] kvm: zapping shadow pages for mmio generation wraparound

## top
Code: 
Tasks: 622 total,  1 running, 621 sleeping,  0 stopped,  0 zombie
%Cpu(s):  3.9 us,  5.7 sy,  0.0 ni, 87.3 id,  3.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  6101852 total,  1346120 used,  4755732 free,  2260 buffers
KiB Swap:  5242876 total,  0 used,  5242876 free.  94500 cached Mem
PID USER  PR  NI  VIRT  RES  SHR S  %CPU %MEM  TIME+ COMMAND
2428 root  20  0 1498964  61220  8224 S  75.6  1.0  1:43.58 kvm
1139 root  1 -19  0  0  0 S  1.3  0.0  0:01.17 z_wr_iss
1385 root  0 -20  0  0  0 S  1.3  0.0  0:01.24 kworker/2:1H
2332 root  20  0  252364  62244  5552 S  1.3  1.0  0:00.73 pvestatd

## kvm_stat
Code: 
kvm statistics
Event  Total Current
kvm_exit  1838410  118390
kvm_entry  1838404  118388
kvm_emulate_insn  1804112  115592
kvm_userspace_exit  1791496  114492
kvm_pio  1791466  114490
kvm_ioapic_set_irq  25846  2241
kvm_set_irq  25846  2241
kvm_pic_set_irq  25846  2241
kvm_inj_virq  16705  1370
kvm_apic  16696  1368
kvm_apic_accept_irq  16696  1368
kvm_eoi  16696  1368
kvm_vcpu_wakeup  14785  1239
kvm_ack_irq  12888  1118
kvm_cpuid  12886  1118
kvm_mmio  60  4
kvm_fpu  34  2
vcpu_match_mmio  30  2
kvm_exit(HLT)  2  0
kvm_exit(APIC_ACCESS)  2  0
kvm_page_fault  2  0
kvm_exit(EXTERNAL_INTERRUPT)  1  0

## tested processors
Code: 
core2duo
pentium
qemu32
athlon - does not boot
host

# guest

## lsmod
Code: 
Module  Size  Used by
piix  9668  0 [parmananet]
This is something that could be of interest

## dmesg
Code: 
piix3: chipset revision 0
piix3: not 100% native mode will probe irqs later
piix3: port 0x01f0 already claimed by ide0
piix3: port 0x0170 already claimed by ide1
piix3: neither ide port neabled (BIOS)
 
That looks normal to me too.

Next thing would be getting deep into Qemu und probably also using kernel debug interface to see what's going on there.

BTW: Have you tried to install a fresh Etch in the VM and encounter the same problems?
 
I just installed it (wow, 5min from vm-bootup to installed-linux-shell) and it works "normally". Everything blazing fast and no guest or host cpu usage at all.
 
Freshly installed debian 4.0r9 works without a problem. The problem I have is the system that has the issues has a custom compiled MySQL server that work's around some issues and I can't just replace it because I don't know what exactly the developers did for this workaround so I need to get it working from the clone.
 
Hmm, but MySQL is not deep inside the kernel such that you will not see whats going on via top/htop.

Could you do a disk and cpu benchmark inside the "fresh" etch and the "old" one? Maybe it'll show at least the bottleneck.
 
  • Like
Reactions: Primoz Cankar
Freshly installed version:
dd if=/dev/zero of=tempfile bs=1b count=1000k
cca 250MB/s

the cloned version is now still booting for almost 2 hours because it's doing a journal recovery on raid devices. I'll update this when I get into the machine. If you have an idea how to test the cpu I would be glad to try it since any modern tst suite is probably not available on debian 4.0 etch
 
Old version: dd write speed cca 120kB/s. I'm now trying to upgrade the system to see if maybe there is some update that was not done that could fix this. Also one weird thing is only 2 modules are listed in lsmod: piix and e1000
 
Last edited:
You must log in or register to reply here.
Top Bottom