After upgrading a node in our cluster tonight to the latest kernel 2.6.32-40-pve, KVM VMs fail to start with the following error:
After seeing this error we rebooted to the older -39 kernel, but it looks like the damage has been done: KVM does not start on this node anymore.
This is catastrophic. What can be done to repair this node?
We tried removing the new kernel package (apt-get remove pve-kernel-2.6.33-40-pve), but it did not help.
Code:
[COLOR=#000000][FONT=tahoma]Could not access KVM kernel module: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]failed to initialize KVM: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK ERROR: start failed: command '/usr/bin/kvm -id 232 -chardev 'socket,id=qmp,path=/var/run/qemu-server/232.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/232.vnc,x509,password -pidfile /var/run/qemu-server/232.pid -daemonize -name havasad -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga cirrus -cpu SandyBridge,+x2apic -m 4096 -k en-us -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:772a175454bc' -drive 'if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/var/lib/vz/images/232/vm-232-disk-1.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap232i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=AE:B3:9A:83:1E:2E,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1[/FONT][/COLOR]After seeing this error we rebooted to the older -39 kernel, but it looks like the damage has been done: KVM does not start on this node anymore.
This is catastrophic. What can be done to repair this node?
We tried removing the new kernel package (apt-get remove pve-kernel-2.6.33-40-pve), but it did not help.
Code:
root@proxmox:/var/log# pveversion -v
proxmox-ve-2.6.32: 3.4-159 (running kernel: 2.6.32-39-pve)
pve-manager: 3.4-8 (running version: 3.4-8/5f8f4e78)
pve-kernel-2.6.32-40-pve: 2.6.32-159
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-39-pve: 2.6.32-157
pve-kernel-2.6.32-33-pve: 2.6.32-138
pve-kernel-2.6.32-37-pve: 2.6.32-150
pve-kernel-2.6.32-34-pve: 2.6.32-140
pve-kernel-2.6.32-38-pve: 2.6.32-155
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-3
pve-cluster: 3.0-18
qemu-server: 3.4-6
pve-firmware: 1.1-4
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-33
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.2-11
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1
Last edited:
