Kernel errors

S

Stephane

New Member
Oct 16, 2008
29
0
1
Hi all, since few days, I met somes errors which stop KVM and containers... The server ping but nothing is accessible like ssh, proxmox web interface and vm...

Linux proxmox 2.6.24-2-pve #1 SMP PREEMPT Wed Jan 14 11:32:49 CET 2009 x86_64 GNU/Linux

proxmox:/var/log# pveversion -v
pve-manager: 1.1-3 (pve-manager/1.1/3718)
qemu-server: 1.0-10
pve-kernel: 2.6.24-5
pve-kvm: 83-1
pve-firmware: 1
vncterm: 0.9-1
vzctl: 3.0.23-1pve1
vzdump: 1.1-1
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1dso1
Click to expand...
/var/log/syslog
Mar 9 12:10:03 proxmox kernel: stack segment: 0000 [1] PREEMPT SMP ---> I lost ssh connection here.
Mar 9 12:10:03 proxmox kernel: CPU: 0
Mar 9 12:10:03 proxmox kernel: Modules linked in: kvm_intel kvm vzethdev vznetdev simfs vzrst vzcpt tun vzdquota vzmon vzdev xt_tcpudp xt_length ipt_ttl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit ipt_tos ipt_REJECT ip_tables x_tables ipv6 bridge dm_snapshot dm_mirror serio_raw fan r8169 snd_hda_intel psmouse intel_agp snd_pcm snd_timer snd_page_alloc snd_hwdep snd thermal button parport_pc parport evdev processor soundcore floppy pcspkr sg scsi_wait_scan dm_mod usbhid hid usb_storage libusual sd_mod sr_mod ide_disk ide_generic ide_cd cdrom ide_core shpchp pci_hotplug uhci_hcd ehci_hcd usbcore iTCO_wdt iTCO_vendor_support i2c_i801 i2c_core ata_piix ahci pata_jmicron pata_acpi ata_generic libata scsi_mod isofs msdos fat
Mar 9 12:10:03 proxmox kernel: Pid: 7239, comm: kvm Not tainted 2.6.24-2-pve #1 ovz005
Mar 9 12:10:03 proxmox kernel: RIP: 0010:[<ffffffff884742c9>] [<ffffffff884742c9>] :kvm:rmap_write_protect+0x59/0x150
Mar 9 12:10:03 proxmox kernel: RSP: 0018:ffff810129955bf8 EFLAGS: 00010206
Mar 9 12:10:03 proxmox kernel: RAX: 00ff000000000000 RBX: 80000000369a5065 RCX: 0000000000000001
Mar 9 12:10:03 proxmox kernel: RDX: ffff81000fc9d118 RSI: ffff81000fc9d118 RDI: ffff8100b9cbfac8
Mar 9 12:10:03 proxmox kernel: RBP: 00ff000000000000 R08: 0000000000000001 R09: 0000000000000021
Mar 9 12:10:03 proxmox kernel: R10: 0000000000000000 R11: ffffffff8849b000 R12: ffff810127930000
Mar 9 12:10:03 proxmox kernel: R13: ffffc20002490498 R14: 0000000000000001 R15: 0000000000008993
Mar 9 12:10:03 proxmox kernel: FS: 00000000420fb960(0000) GS:ffffffff8060b000(0000) knlGS:0000000000000000
Mar 9 12:10:03 proxmox kernel: CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
Mar 9 12:10:03 proxmox kernel: CR2: 0000000000250000 CR3: 0000000127dd7000 CR4: 00000000000026e0
Mar 9 12:10:03 proxmox kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 9 12:10:03 proxmox kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 9 12:10:03 proxmox kernel: Process kvm (pid: 7239, veid=0, threadinfo ffff810129954000, task ffff8101299688f0)
Mar 9 12:10:03 proxmox kernel: Stack: ffff8101284d4340 ffff81012796c760 ffff810008df5210 0000000000000000
Mar 9 12:10:03 proxmox kernel: 0000000000000001 0000000000008993 ffff81012796c000 ffffffff884747ad
Mar 9 12:10:03 proxmox kernel: 0000000000000202 0000000088468d68 ffff810127931798 0000000000000000
Mar 9 12:10:03 proxmox kernel: Call Trace:
Mar 9 12:10:03 proxmox kernel: [<ffffffff884747ad>] :kvm:kvm_mmu_get_page+0x2ed/0x390
Mar 9 12:10:03 proxmox kernel: [<ffffffff88476bd4>] :kvm:paging64_page_fault+0x294/0x440
Mar 9 12:10:03 proxmox kernel: [<ffffffff8849a91d>] :kvm_intel:vmx_set_cr3+0x4d/0x110
Mar 9 12:10:03 proxmox kernel: [<ffffffff88474cf9>] :kvm:kvm_mmu_page_fault+0x19/0x90
Mar 9 12:10:03 proxmox kernel: [<ffffffff8846f521>] :kvm:kvm_arch_vcpu_ioctl_run+0x3a1/0x8c0
Mar 9 12:10:03 proxmox kernel: [<ffffffff8026cdc5>] do_futex+0x595/0xbf0
Mar 9 12:10:03 proxmox kernel: [<ffffffff88469fa5>] :kvm:kvm_vcpu_ioctl+0x525/0x5b0
Mar 9 12:10:03 proxmox kernel: [<ffffffff804a5a82>] thread_return+0x3d/0x5bb
Mar 9 12:10:03 proxmox kernel: [<ffffffff8020a8ba>] __switch_to+0x1ba/0x320
Mar 9 12:10:03 proxmox kernel: [<ffffffff802d502f>] do_ioctl+0x2f/0xa0
Mar 9 12:10:03 proxmox kernel: [<ffffffff802d5114>] vfs_ioctl+0x74/0x2d0
Mar 9 12:10:03 proxmox kernel: [<ffffffff802d53b9>] sys_ioctl+0x49/0x80
Mar 9 12:10:03 proxmox kernel: [<ffffffff80257220>] sys_clock_gettime+0x80/0xc0
Mar 9 12:10:03 proxmox kernel: [<ffffffff8020c4ee>] system_call+0x7e/0x83
Mar 9 12:10:03 proxmox kernel:
Mar 9 12:10:03 proxmox kernel:
Mar 9 12:10:03 proxmox kernel: Code: 48 8b 5d 00 f6 c3 01 0f 84 de 00 00 00 48 89 df e8 e2 d3 ff
Mar 9 12:10:03 proxmox kernel: RIP [<ffffffff884742c9>] :kvm:rmap_write_protect+0x59/0x150
Mar 9 12:10:03 proxmox kernel: RSP <ffff810129955bf8>
Mar 9 12:10:03 proxmox kernel: ---[ end trace e874beb7c7ab95e8 ]---
Mar 9 12:10:03 proxmox kernel: note: kvm[7239] exited with preempt_count 2
Click to expand...
/var/log/messages (a lot of)
Mar 9 11:57:38 proxmox kernel: sr 0:0:1:0: [sr0] Device not ready: Sense Key : Not Ready [current]
Mar 9 11:57:38 proxmox kernel: sr 0:0:1:0: [sr0] Device not ready: Add. Sense: Medium not present
Click to expand...
/var/log# fdisk -l
Disk /dev/sda: 250.0 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 66 524288 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 66 30401 243671712 8e Linux LVM

Disk /dev/dm-0: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 62.2 GB, 62277025792 bytes
255 heads, 63 sectors/track, 7571 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/dm-2: 178.6 GB, 178656378880 bytes
255 heads, 63 sectors/track, 21720 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-2 doesn't contain a valid partition table

Disk /dev/dm-3: 178.6 GB, 178656378880 bytes
255 heads, 63 sectors/track, 21720 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-3 doesn't contain a valid partition table

Disk /dev/dm-4: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Click to expand...
proxmox:/var/log# mount
/dev/pve/root on / type ext3 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/mapper/pve-data on /var/lib/vz type ext3 (rw)
/dev/sda1 on /boot type ext3 (rw)
Click to expand...
proxmox:/var/log# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 232.38 GB / not usable 0
Allocatable yes
PE Size (KByte) 4096
Total PE 59490
Free PE 767
Allocated PE 58723
PV UUID Ss56sQ-sWI4-Za7c-emHS-1wIf-KMlv-Jabo17
Disk /dev/dm-4 doesn't contain a valid partition table
Click to expand...
So, does it a kernel and/or disk problem?

Than youk for any help.

Stephane.
 
Last edited:
Please can you post your VM configuration (/etc/qemu-server/VMID.conf)?

Is the bug reproducable somehow?

-Dietmar
 
dietmar said:
Please can you post your VM configuration (/etc/qemu-server/VMID.conf)?

Is the bug reproducable somehow?

-Dietmar
Click to expand...

The bug appear each time à boot the server, so it is unsable in production...

All VMs are impacted, the OS freeze and I need to reboot...

proxmox:/etc/qemu-server# cat 104.conf
name: xp_pro
ide2: cdrom,media=cdrom
smp: 1
vlan0: rtl8139=92:F9:5A:4E:FA:5B
bootdisk: ide0
ide0: vm-104-disk.qcow2
ostype: wxp
memory: 256
onboot: 1
boot: cad
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1

proxmox:/etc/qemu-server# cat 105.conf
name: xubuntu
ide2: none,media=cdrom
smp: 1
vlan0: rtl8139=6E:47:C8:19:44:14
bootdisk: ide0
ide0: vm-105-disk.qcow2
ostype: l26
memory: 256
onboot: 1
boot: c
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1

proxmox:/etc/qemu-server# cat 106.conf
name: Windows7
ide2: en_windows_7_beta_dvd_x86_x15-29073.iso,media=cdrom
smp: 1
vlan0: rtl8139=8E:7C:B0:65:F9:01
bootdisk: ide0
ide0: vm-106-disk.qcow2
ostype: w2k8
memory: 512
onboot: 1

proxmox:/etc/qemu-server# cat 108.conf
name: w2k3
ide2: none,media=cdrom
smp: 1
bootdisk: ide0
ide0: vm-108-disk.qcow2
ostype: w2k3
memory: 256
onboot: 1
boot: cd
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
vlan0: virtio=A2:05:8F:AB:EE:93,rtl8139=8A:69:06:3D:EA:4A

proxmox:/etc/qemu-server# cat 109.conf
name: w2k3-dev
ide2: cdrom,media=cdrom
smp: 1
vlan0: virtio=DE:E6:1B:1E:12:CC
bootdisk: ide0
ide0: vm-109-disk.qcow2
ostype: w2k3
memory: 256
onboot: 1

proxmox:/etc/qemu-server# cat 112.conf
name: ubuntu
ide2: none,media=cdrom
smp: 1
vlan0: rtl8139=B2:55:E3:54:A1:27
bootdisk: ide0
ide0: vm-112-disk.qcow2
ostype: l26
memory: 256
onboot: 0

proxmox:/etc/qemu-server# cat 116.conf
name: switchvox
ide2: digium-switchvox-free_8634.iso,media=cdrom
smp: 1
vlan0: rtl8139=C6:2A:5A:4B:DA:35
bootdisk: ide0
ide0: vm-116-disk.qcow2
ostype: other
memory: 256
onboot: 1

proxmox:/etc/qemu-server# cat 117.conf
name: trixbox
ide2: trixbox-2.6.2.2.iso,media=cdrom
smp: 1
vlan0: rtl8139=76:33:00:94:8A:D9
bootdisk: ide0
ide0: vm-117-disk.qcow2
ostype: other
memory: 512
onboot: 1

proxmox:/etc/qemu-server# cat 118.conf
name: voip
ide2: AsteriskNOW-1.0.2.1-x86-disc1.iso,media=cdrom
smp: 1
vlan0: rtl8139=06:60:80:05:C0:42
bootdisk: ide0
ide0: vm-118-disk.qcow2
ostype: l26
memory: 512
onboot: 1

proxmox:/etc/qemu-server# cat 119.conf
name: w2k8
ide2: Windows_Server_2008_datacenter_enterprise_standard_x64.iso,media=cdrom
smp: 1
vlan0: rtl8139=56:18:5A:DC:C8:24
ide0: vm-119-disk.qcow2
ostype: w2k8
memory: 512
onboot: 1
boot: da
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
Click to expand...
 
Last edited:
Stephane said:
The bug appear each time à boot the server, so it is unsable in production...

All VMs are impacted, the OS freeze and I need to reboot...
Click to expand...

Do you use software RAID? Is this a Proxmox VE server installed from OVH?
 
This does not happen on a standard install. So do you changed anything? If not, please test if the RAM/HW is OK.
 
dietmar said:
whats the output of 'vgs' and 'lvs'
Click to expand...

proxmox:~# vgs
VG #PV #LV #SN Attr VSize VFree
pve 1 3 1 wz--n- 232.38G 3.00G

proxmox:~# lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
data pve owi-ao 166.39G
root pve -wi-ao 58.00G
swap pve -wi-ao 4.00G
vzsnap pve Swi-I- 1.00G data 100.00
 
Well, the problem seems to be from disk.

So, I have install a fresh version of Proxmox PVE on a new disk and I want to restore all the VMs from te old disk to tje new one and I met some problem regarding to the LVM names:

When I do a "vgchange -an pve" I get a "Found more than one VG called pve. Please supply VG uuid".

So, vgdisplay give me the uuid of the old LVM from the defeckt disk but I met still problem regarding to rename the old LVM with "vgrename uuid pve_old" :

"Volume group "pve" still has active LVs."

I do a "lvchange -an uuid: "Volume group "uuid" not found"...

:(
 
Last edited:
Stephane said:
Well, the problem seems to be from disk.

So, I have install a fresh version of Proxmox PVE on a new disk and I want to restore all the VMs from te old disk to tje new one and I met some problem regarding to the LVM names:
Click to expand...

You attached both drives to the computer (old and new)? That does not work because both drives contain LVM volumes with the same name.
 
Thank you for your quick sunday morning answer :)

dietmar said:
You attached both drives to the computer (old and new)? That does not work because both drives contain LVM volumes with the same name.
Click to expand...

Yes. Is it really impossible to do a backup from and to the same server? Is there no way to do this?
 
You must log in or register to reply here.
Top Bottom