Crashes and Kernel Panic

[evg32]

At first I installed ProxMox 3.4-11 on my new server (i7-6700, 32GB Ram) at Hetzner. It worked nice for several days. When I started to add and launch new VMs (Windows XP) it begun crashing:


Previous pveversion:

# pveversion -v
proxmox-ve-2.6.32: 3.4-166 (running kernel: 2.6.32-43-pve)
pve-manager: 3.4-11 (running version: 3.4-11/6502936f)
pve-kernel-2.6.32-43-pve: 2.6.32-166
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-3
pve-cluster: 3.0-19
qemu-server: 3.4-6
pve-firmware: 1.1-5
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-34
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: not correctly installed
vzquota: 3.1-2
pve-qemu-kvm: 2.2-14
ksm-control-daemon: not correctly installed
glusterfs-client: 3.5.2-1

Current pveversion:

# pveversion -v
proxmox-ve: 4.1-30 (running kernel: 4.2.6-1-pve)
pve-manager: 4.1-4 (running version: 4.1-4/ccba54b0)
pve-kernel-4.2.6-1-pve: 4.2.6-30
pve-kernel-4.2.2-1-pve: 4.2.2-16
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 0.17.2-1
pve-cluster: 4.0-30
qemu-server: 4.0-43
pve-firmware: 1.1-7
libpve-common-perl: 4.0-42
libpve-access-control: 4.0-10
libpve-storage-perl: 4.0-38
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.4-18
pve-container: 1.0-36
pve-firewall: 2.0-14
pve-ha-manager: 1.0-16
ksm-control-daemon: not correctly installed
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-5
lxcfs: 0.13-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1

Then I upgraded kernel from 2.6.32-43-pve to 4.2.6-1 but it didn't solve the issue.

Where should I dig in to solve the problem and get the server stable?

 

[windinternet]

So, what settings did you use for your Windows XP kvms? Emulated hardware/processor? Are you installing XP 64-bit or 32-bit?

 

[fireon]

Setting up also 2 XPs 32bit with KVM and Virtio, never had any problems before. (PVE4)

 

[evg32]

Setting up also 2 XPs 32bit with KVM and Virtio, never had any problems before. (PVE4)

7 WinXP VMs were working fine, but when I added 8th it begun crashing.
I have 20 WinXP VMs on PVE version: 3.4-11 and they work nice.
But my new server doesn't work.

 

[evg32]

So, what settings did you use for your Windows XP kvms? Emulated hardware/processor? Are you installing XP 64-bit or 32-bit?

The CPU is Default (kvm64) and Windows XP is 32-bit.

I have a couple of servers with Xeon CPU E3-1246 v3 and PVE version: 3.4-11. They work fine, but a new one with i7-6700 doesn't work correctly.

Also I noticed ACPI Errors and "CPU stuck for..."

And it crashes now when I'm trying to clone a VM. All VMs are shutdown.

 

[aderumier]

Is the server bios and cpu microcode update on last version ?

 

[evg32]

The hoster says they don't have a BIOS update yet. i7-6700 was released 4 month ago.

 

[fireon]

The CPU is Default (kvm64) and Windows XP is 32-bit.

I have a couple of servers with Xeon CPU E3-1246 v3 and PVE version: 3.4-11. They work fine, but a new one with i7-6700 doesn't work correctly.

Also I noticed ACPI Errors and "CPU stuck for..."

And it crashes now when I'm trying to clone a VM. All VMs are shutdown.

Xeon much more features then an i7. Maybe in ths case this is an problem. We use always Xeon for prodction virtualisation.

 

[windinternet]

Did you use ZFS in the Proxmox installer?

 

[evg32]

I used Soft Raid level 1 and lvm.
Something like this:
PART /boot ext3 512M
PART lvm vg0 all
LV vg0 root / ext4 20G
LV vg0 swap swap swap 16G

 

[evg32]

Xeon much more features then an i7. Maybe in ths case this is an problem. We use always Xeon for prodction virtualisation.

Well, the CPU Proxmox requirements are:

CPU: 64bit (Intel EMT64 or AMD64)
Intel VT/AMD-V

I think any i7 CPU would be fine.

 

[windinternet]

It seems an odd coincidence that the number of XP's you can run hits a max at 7, if you have a processor with 8 hypertreads. You probably set one cpu per vm, and hardware acceleration enabled, right? Maybe you could try and disable hardware acceleration for a moment.

 

[evg32]

Trying to re-install a fresh PVE 3.4-11...

Yep, the KVM Hardware Virtualization is enabled. The max number of 7 instances doesn't look realistic for me. CPU is a shared resource.
Xeon E3-1246 has the same max number of thread as i7, but works nice at 20+ VMs.

 

[windinternet]

If your limit is consistent at 7 hardware accelerated processors committed in KVM, then that seems the point to start looking.

 

[evg32]

Where can I change this limit?
I can only see KVM Hardware Acceleration for each VM via web interface.

 

[windinternet]

@evg32,

Sorry, I meant, IF that is what you are experiencing. There is no actual setting for this. But what happens if you give one machine "four" cpus, can you then only start three extra vm's with one cpu?

If there is this "hard" limit on 7 vcpus, and given the type of processor in your system, then that would be a good place to start troubleshooting.