KVM will not start after upgrade to 1.6.

thump

New Member
Jun 14, 2010
5
0
1
Haven't been able to get any KVM to start after upgrading to 1.6 and proxmox-ve-2.6.32. Tried creating a new KVM for testing:

/usr/sbin/qm start 103
warning: could not open /dev/net/tun: no virtual network emulation
start failed: command '/usr/bin/kvm -monitor unix:/var/run/qemu-server/103.mon,server,nowait -vnc unix:/var/run/qemu-server/103.vnc,password -pidfile /var/run/qemu-server/103.pid -daemonize -usbdevice tablet -name mm-tangle1 -smp sockets=1,cores=1 -nodefaults -boot menu=on,order=c -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/untangle_720_32bit.iso,if=ide,index=2,media=cdrom -drive file=/var/lib/vz/images/103/vm-103-disk-1.raw,if=virtio,index=0,boot=on -m 1024 -net tap,vlan=1,ifname=vmtab103i1,script=/var/lib/qemu-server/bridge-vlan -net nic,vlan=1,model=virtio,macaddr=BA:B5:32:D8:2E:E2' failed with exit code 1
VM 103 start failed -

# pveversion -v
pve-manager: 1.6-2 (pve-manager/1.6/5087)
running kernel: 2.6.32-3-pve
proxmox-ve-2.6.32: 1.6-12
pve-kernel-2.6.32-3-pve: 2.6.32-12
pve-kernel-2.6.24-10-pve: 2.6.24-21
qemu-server: 1.1-18
pve-firmware: 1.0-6
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.24-1pve3
vzdump: 1.2-7
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.5-1
ksm-control-daemon: 1.0-4

# uname -a
Linux mm-prox0 2.6.32-3-pve #1 SMP Thu Sep 2 09:18:16 CEST 2010 x86_64 GNU/Linux

Problem appears that there is no "tun" module available with the new kernel.
 
Further info after another reboot. Appears there is no /dev/kvm.

/usr/sbin/qm start 101
open /dev/kvm: No such file or directory
Could not initialize KVM, will disable KVM support
warning: could not open /dev/net/tun: no virtual network emulation
start failed: command '/usr/bin/kvm -monitor unix:/var/run/qemu-server/101.mon,server,nowait -vnc unix:/var/run/qemu-server/101.vnc,password -pidfile /var/run/qemu-server/101.pid -daemonize -usbdevice tablet -name mm-tangle0 -smp sockets=1,cores=1 -nodefaults -boot menu=on,order=c -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/untangle_720_32bit.iso,if=ide,index=2,media=cdrom -drive file=/var/lib/vz/images/101/vm-101-disk-1.raw,if=virtio,index=0,boot=on -m 1024 -net tap,vlan=1,ifname=vmtab101i1,script=/var/lib/qemu-server/bridge-vlan -net nic,vlan=1,model=virtio,macaddr=1A:3B:5A:7F:F8:F4 -net tap,vlan=2,ifname=vmtab101i2,script=/var/lib/qemu-server/bridge-vlan -net nic,vlan=2,model=virtio,macaddr=A2:79:FE:B8:55:7A -net tap,vlan=3,ifname=vmtab101i3,script=/var/lib/qemu-server/bridge-vlan -net nic,vlan=3,model=virtio,macaddr=7A:F2:2F:1D:86:41' failed with exit code 1
VM 101 start failed -
 
Last update. This thread should be re-titled "KVM will not start after upgrade to 2.6.32-3-pve". I had upgraded both the pve and the kernel at the same time.

After a reboot back into the 2.6.24-10-pve kernel everything is working fine with pve-manager 1.6. Guess I'll have to hold off on the new kernel for a bit.
 
works here. any hint in dmesg? KVM module not loaded?

give also details about your mainboard/cpu for reference.
 
I just finished reinstalling Proxmox & after qmrestores found the same issue exists here.
Changing kernels didn't make a difference.

It might be worth noting that I re-installed from the 1.5.4627-3 iso, since at the time we were still using the previous version. The upgrade was released while I was amidst the qmrestore process, so I simply did aptitude full-upgrade before & after.

any hint in dmesg?
I didn't see anything that appeared out of the ordinary in dmesg either time I tried.

give also details about your mainboard/cpu for reference.
Hopefully the following is sufficient, lshw outputs 747 lines.

Mainboard/CPU:
Dell Poweredge 2950, 2x quad-core Intel Xeon X5470 CPU

Attempt 1 with KSM:
Code:
bascule:~# pveversion -v
pve-manager: 1.6-2 (pve-manager/1.6/5087)
running kernel: 2.6.32-2-pve
proxmox-ve-2.6.32: 1.6-13
pve-kernel-2.6.32-3-pve: 2.6.32-13
pve-kernel-2.6.18-2-pve: 2.6.18-5
qemu-server: 1.1-18
pve-firmware: 1.0-7
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-7
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.5-1
ksm-control-daemon: 1.0-4
KVM module is loaded:
Code:
bascule:~# lsmod
Module                  Size  Used by
tun                    12299  0
kvm_intel              37970  0
kvm                   213288  1 kvm_intel
vzethdev                7189  0
vznetdev               17967  0
simfs                   2991  0
vzrst                 108126  0
vzcpt                  95234  0
vzdquota               34774  0 [permanent]
vzmon                  16237  4 vzethdev,vznetdev,vzrst,vzcpt
vzdev                   1824  4 vzethdev,vznetdev,vzdquota,vzmon
xt_tcpudp               2319  0
xt_length               1164  0
xt_hl                   1313  0
xt_tcpmss               1401  0
xt_TCPMSS               2935  0
iptable_mangle          2881  0
iptable_filter          2322  0
xt_multiport            2267  0
xt_limit                1782  0
xt_dscp                 1805  0
ipt_REJECT              1953  0
ip_tables              14107  2 iptable_mangle,iptable_filter
x_tables               13117  10 xt_tcpudp,xt_length,xt_hl,xt_tcpmss,xt_TCPMSS,xt_multiport,xt_limit,xt_dscp,ipt_REJECT,ip_tables
ib_iser                24210  0
rdma_cm                20486  1 ib_iser
ib_cm                  26192  1 rdma_cm
iw_cm                   6275  1 rdma_cm
ib_sa                  16199  2 rdma_cm,ib_cm
ib_mad                 29825  2 ib_cm,ib_sa
ib_core                40679  6 ib_iser,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad
ib_addr                 4389  1 rdma_cm
iscsi_tcp               8017  0
libiscsi_tcp           10898  1 iscsi_tcp
libiscsi               29206  3 ib_iser,iscsi_tcp,libiscsi_tcp
scsi_transport_iscsi    24844  4 ib_iser,iscsi_tcp,libiscsi
bridge                 40246  0
stp                     1440  1 bridge
btusb                   9833  0
snd_pcm                60487  0
snd_timer              15550  1 snd_pcm
i5k_amb                 4343  0
snd                    46398  2 snd_pcm,snd_timer
soundcore               4598  1 snd
snd_page_alloc          6137  1 snd_pcm
bluetooth              41635  1 btusb
evdev                   7368  0
serio_raw               3768  0
pcspkr                  1699  0
rng_core                2990  0
rfkill                 12868  1 bluetooth
dcdbas                  5000  0
psmouse                49681  0
i5000_edac              7188  0
button                  4682  0
edac_core              29261  3 i5000_edac
processor              30111  8
shpchp                 26264  0
pci_hotplug            21219  1 shpchp
ext3                  105398  3
jbd                    36509  1 ext3
mbcache                 5050  1 ext3
dm_mirror              10827  0
dm_region_hash          6584  1 dm_mirror
dm_log                  7381  2 dm_mirror,dm_region_hash
dm_snapshot            18289  0
dm_mod                 53194  12 dm_mirror,dm_log,dm_snapshot
usbhid                 32892  0
hid                    62681  1 usbhid
sg                     18664  0
sr_mod                 12682  0
cdrom                  29415  1 sr_mod
ses                     5128  0
sd_mod                 29585  3
crc_t10dif              1276  1 sd_mod
enclosure               5407  1 ses
ata_piix               21028  0
ata_generic             2983  0
ehci_hcd               30543  0
uhci_hcd               18393  0
usbcore               121538  5 btusb,usbhid,ehci_hcd,uhci_hcd
nls_base                6377  1 usbcore
libata                132864  2 ata_piix,ata_generic
ixgbe                 105998  0
megaraid_sas           29004  2
dca                     3761  1 ixgbe
mdio                    3000  1 ixgbe
scsi_mod              121461  10 ib_iser,iscsi_tcp,libiscsi,scsi_transport_iscsi,sg,sr_mod,ses,sd_mod,libata,megaraid_sas
bnx2                   57177  0
thermal                11754  0
fan                     3362  0
thermal_sys            11942  3 processor,thermal,fan
bascule:~#
Start Failed syslog message:
Code:
Sep  4 13:20:41 bascule qm[3638]: VM 209 start failed: command  '/usr/bin/kvm -monitor unix:/var/run/qemu-server/209.mon,server,nowait  -vnc unix:/var/run/qemu-server/209.vnc,password -pidfile  /var/run/qemu-server/209.pid -daemonize -usbdevice tablet -name Rohini  -smp sockets=1,cores=4 -nodefaults -boot menu=on,order=dc -vga cirrus  -tdf -localtime -k en-us -drive if=ide,index=2,media=cdrom -drive  file=/var/lib/vz/images/209/vm-209-disk-1.raw,if=virtio,index=0,boot=on  -drive file=/var/lib/vz/images/209/vm-209-disk-2.raw,if=virtio,index=1  -m 4096 -net  tap,vlan=1,ifname=vmtab209i1,script=/var/lib/qemu-server/bridge-vlan  -net nic,vlan=1,model=virtio,macaddr=A2:47:D6:89:D2:28' failed with exit  code 1
Sep  4 13:20:41 bascule pvedaemon[3637]: VM 209 start failed -
Attempt 2 without KSM:
Code:
bascule:~# pveversion -v
pve-manager: 1.6-2 (pve-manager/1.6/5087)
running kernel: 2.6.24-12-pve
proxmox-ve-2.6.24: 1.5-24
pve-kernel-2.6.18-2-pve: 2.6.18-5
pve-kernel-2.6.24-12-pve: 2.6.24-24
qemu-server: 1.1-18
pve-firmware: 1.0-7
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-7
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.5-1
KVM module is loaded:
Code:
bascule:~# lsmod
Module                  Size  Used by
kvm_intel              58568  0
kvm                   207496  1 kvm_intel
vzethdev               23552  0
vznetdev               32904  0
simfs                  14064  0
vzrst                 158248  0
vzcpt                 131256  0
tun                    23168  2 vzrst,vzcpt
vzdquota               60016  0 [permanent]
vzmon                  58008  4 vzethdev,vznetdev,vzrst,vzcpt
vzdev                  12808  4 vzethdev,vznetdev,vzdquota,vzmon
xt_tcpudp              12160  0
xt_length              10752  0
ipt_ttl                10624  0
xt_tcpmss              11008  0
xt_TCPMSS              13440  0
iptable_mangle         13824  0
iptable_filter         13568  0
xt_multiport           12160  0
xt_limit               11904  0
ipt_tos                10368  0
ipt_REJECT             13824  0
ip_tables              33384  2 iptable_mangle,iptable_filter
x_tables               34056  10 xt_tcpudp,xt_length,ipt_ttl,xt_tcpmss,xt_TCPMSS,xt_multiport,xt_limit,ipt_tos,ipt_REJECT,ip_tables
ipv6                  350592  49 vzrst,vzcpt,vzmon
ib_iser                47112  0
rdma_cm                44788  1 ib_iser
ib_cm                  45864  1 rdma_cm
iw_cm                  19848  1 rdma_cm
ib_sa                  34912  2 rdma_cm,ib_cm
ib_mad                 52008  2 ib_cm,ib_sa
ib_core                71552  6 ib_iser,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad
ib_addr                17672  1 rdma_cm
iscsi_tcp              36992  0
libiscsi               38912  2 ib_iser,iscsi_tcp
scsi_transport_iscsi    44560  4 ib_iser,iscsi_tcp,libiscsi
bridge                 75304  0
hci_usb                27036  0
serio_raw              16388  0
dcdbas                 18736  0
i5000_edac             19464  0
pcspkr                 12160  0
bluetooth              76324  1 hci_usb
psmouse                54684  0
evdev                  22912  0
shpchp                 46236  0
edac_core              61840  3 i5000_edac
pci_hotplug            43056  1 shpchp
button                 18080  0
dm_mirror              34816  0
dm_snapshot            27744  0
dm_mod                 80248  9 dm_mirror,dm_snapshot
usbhid                 43616  0
hid                    53312  1 usbhid
sg                     49560  0
sr_mod                 27812  0
cdrom                  49064  1 sr_mod
ata_piix               32004  0
pata_acpi              17152  0
sd_mod                 41088  3
ehci_hcd               50188  0
ata_generic            17156  0
uhci_hcd               38176  0
libata                189616  3 ata_piix,pata_acpi,ata_generic
megaraid_sas           45884  2
usbcore               180912  5 hci_usb,usbhid,ehci_hcd,uhci_hcd
scsi_mod              189752  9 ib_iser,iscsi_tcp,libiscsi,scsi_transport_iscsi,sg,sr_mod,sd_mod,libata,megaraid_sas
bnx2                  205068  0
thermal                26912  0
processor              49096  9 thermal
fan                    13960  0
bascule:~#
Start Failed syslog message:
Code:
Sep  4 13:40:09 bascule qm[5797]: VM 209 start failed: command  '/usr/bin/kvm -monitor unix:/var/run/qemu-server/209.mon,server,nowait  -vnc unix:/var/run/qemu-server/209.vnc,password -pidfile  /var/run/qemu-server/209.pid -daemonize -usbdevice tablet -name Rohini  -smp sockets=1,cores=4 -nodefaults -boot menu=on,order=dc -vga cirrus  -tdf -localtime -k en-us -drive if=ide,index=2,media=cdrom -drive  file=/var/lib/vz/images/209/vm-209-disk-1.raw,if=virtio,index=0,boot=on  -drive file=/var/lib/vz/images/209/vm-209-disk-2.raw,if=virtio,index=1  -m 4096 -net  tap,vlan=1,ifname=vmtab209i1,script=/var/lib/qemu-server/bridge-vlan  -net nic,vlan=1,model=virtio,macaddr=A2:47:D6:89:D2:28' failed with exit  code 1
Sep  4 13:40:09 bascule pvedaemon[5796]: VM 209 start failed -
 
same for me first it look for modules at wrong place
FATAL: Could not load /lib/modules/2.6.32-1-pve/modules.dep: No such file or directory

i made symlink for 32-3 with 32-1 then there are some errors with module tun
 
Sep 4 21:20:22 genabaris modprobe: FATAL: Error inserting tun (/lib/modules/2.6.32-3-pve/kernel/drivers/net/tun.ko): Invalid module format
Sep 4 21:20:22 genabaris kernel: tun: disagrees about version of symbol module_layout
 
Sep 4 21:20:22 genabaris modprobe: FATAL: Error inserting tun (/lib/modules/2.6.32-3-pve/kernel/drivers/net/tun.ko): Invalid module format
Sep 4 21:20:22 genabaris kernel: tun: disagrees about version of symbol module_layout

what is the output of:

# md5sum /lib/modules/2.6.32-3-pve/kernel/drivers/net/tun.ko

and

# md5sum /boot/System.map-2.6.32-3-pve
 
Last edited:
Sorry for the delayed response. I was out for the holiday and then playing catch up at work.

* Mainboard/CPU: ASUS RS100-E5/PI2 1U Barebone Server (P5BV-M/RS100-E5 mobo) / Intel Xeon X3220 Kentsfield 2.4GHz LGA 775 Quad-Core (SLACT, G0 stepping)

* dmesg doesn't produce anything exciting.

* /lib/modules/2.6.32-1-pve/modules.dep is an empty file.

* modprobe of kvm, tun, ext2 = not found

* md5sum /lib/modules/2.6.32-3-pve/kernel/drivers/net/tun.ko
b43bc85fbb9e067435bc5e26a88aa88d

* md5sum /boot/System.map-2.6.32-3-pve
I can't mount /boot because mine is ext2 and the running kernel (2.6.32-3-pve) won't load ext2 module (not found).

Please let me know if you need further info. I should be around all afternoon for testing.
 
Booted back into 2.6.24-10-pve and did an apt-get upgrade to the new replacement 2.6.32-3-pve. All now seems to be working with 2.6.32-3-pve. Thanks!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!