New 2.6.32 Kernel in pvetest (pve-kernel-2.6.32-6-pve: 2.6.32-50)

[martin]

We just released a new kernel to the pvetest repository.

Release notes:

- pve-kernel-2.6.32-6-pve (2.6.32-50)

• update to vzkernel-2.6.32-042stab039.10.src.rpm
• update e1000e to 1.6.3
• update igb to 3.2.10
• update ixgbe to 3.6.7

__________________
Best regards,
Martin Maurer

 

[udo]

Hi,
will this kernel also used on pve2? Because my testmachines are all pve2 at this time.

Udo

 

[tom]

think so, yes. we plan to release updates to 2.0 beta (also new iso) soon, including the backup/restore (with gui).

 

[SuSt]

Dear developers.

Do you publish anywhere information about changes, implemented in your regular release? I am interested in full change log and would like to see a full list of patches that you have applied to "original" RHEL kernel.

For example, I would like to know if this bugfix is included in this build, instead of testing it again and again.

Since a moment, ProxMox team stopped to upload src.deb packets with kernel sources into repo. Or maybe I fail to find them. Please, provide more information about the build or publish sources, for our convenience. Then we will be able to test new kernels more efficiently. Thank you.

 

[tom]

Dear developers.

Do you publish anywhere information about changes, implemented in your regular release? I am interested in full change log and would like to see a full list of patches that you have applied to "original" RHEL kernel.

For example, I would like to know if this bugfix is included in this build, instead of testing it again and again.

as posted, the kernel is based on vzkernel-2.6.32-042stab039.10.src.rpm. so as the fix you mentioned should be already in 2.6.32-042stab039.2, it should be also in.

Since a moment, ProxMox team stopped to upload src.deb packets with kernel sources into repo. Or maybe I fail to find them. Please, provide more information about the build or publish sources, for our convenience. Then we will be able to test new kernels more efficiently. Thank you.

we never uploaded src.deb packages, just as tgz. we started to maintain git.proxmox.com (but for 2.x releases only). I assume this will help in future.

 

[SuSt]

OK. Concerning the subject, 2.6.32-50.

The bug I mentioned above (number 2045 in OpenVZ bugzilla) is still present here too. And one more issue.

I have updated the kernel from previous version, then successfully rebooted into it. Then I decided to reboot once again. My box has not come up after it. I can't tell exactly what happened, because I've lost remote control of the machine. The last message I've seen on my remote vty was something like "Will now restart". And then blackout followed.

 

[stef1777]

Same problem here. I was running -49 on a test server.

I installed -50 and rebooted the server (PowerEdge 2950). I lost him. Will have to see exactly was happening monday as I don't have a remote control board on this old server.

 

[tom]

OK. Concerning the subject, 2.6.32-50.

The bug I mentioned above (number 2045 in OpenVZ bugzilla) is still present here too.

looks like the openvz team does not include it? ask them why - or did you asked already (using another username)?

And one more issue.

I have updated the kernel from previous version, then successfully rebooted into it. Then I decided to reboot once again. My box has not come up after it. I can't tell exactly what happened, because I've lost remote control of the machine. The last message I've seen on my remote vty was something like "Will now restart". And then blackout followed.

would be nice if you find out what happened here. our boxes runs well so far with this kernel.

 

[SuSt]

or did you asked already (using another username)?

Yes, already asked. My name is "Stanislav" on their bugtracker. They provided no answer.

would be nice if you find out what happened here.

I can't answer exactly too. I came by feet to the problematic server on monday. Connected display and keyboard to it and found just a blank screen with nothing on it. Server did not respond either on my actions or to serial RS232 vty, so I pressed "reset" button.

The most wonderful thing is that I rebooted it in normal way (issuing "reboot" command on console) several times after this incident, and everything went allright. But I would not like to repeat this experiment again very much, because I was forced to do the things I haven't planned at all.

 

[stef1777]

Same problem here. I was running -49 on a test server.

I installed -50 and rebooted the server (PowerEdge 2950). I lost him. Will have to see exactly was happening monday as I don't have a remote control board on this old server.

Hello!

I've found the server completely crashed (no keyboard, screen frozen). Seems to be a problem during stop process with KVM.






pve-manager: 1.9-24 (pve-manager/1.9/6542)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 1.9-47
pve-kernel-2.6.32-6-pve: 2.6.32-50
qemu-server: 1.1-32
pve-firmware: 1.0-14
libpve-storage-perl: 1.0-19
vncterm: 0.9-2
vzctl: 3.0.29-2pve1
vzdump: 1.2-16
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.0-6

(was kernel -49 during server restart)

[EDIT]

Seems related to kernel -49. I've no more the problem with -50.

I've one running KVM VE on the server. When trying to restart the proxmox server with restart command, the KVM VE stop process don't go well and crash. I've found this in syslog (many time for each cpu):

Nov 6 04:07:01 vstest-01 kernel: CPU 7:
Nov 6 04:07:01 vstest-01 kernel: Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm simfs nf_conntrack(-) nf_defrag_ipv4 nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota vzdev ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit xt_dscp ipt_REJECT ip_tables vzevent ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi bridge stp llc snd_pcsp snd_pcm snd_timer snd soundcore dcdbas snd_page_alloc tpm_tis tpm serio_raw tpm_bios ghes i5000_edac edac_core i5k_amb hwmon hed shpchp ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot sg floppy ata_piix ata_generic pata_acpi bnx2 megaraid_sas tg3 [last unloaded: nf_conntrack_ipv4]
Nov 6 04:07:01 vstest-01 kernel: Pid: 67060, comm: kvm Tainted: G W ---------------- 2.6.32-6-pve #1 042stab039_5 PowerEdge 2950
Nov 6 04:07:01 vstest-01 kernel: RIP: 0010:[<ffffffffa0542289>] [<ffffffffa0542289>] kvm_handle_fault_on_reboot+0x19/0x30 [kvm]
Nov 6 04:07:01 vstest-01 kernel: RSP: 0018:ffff880119b69be8 EFLAGS: 00000202
Nov 6 04:07:01 vstest-01 kernel: RAX: ffff880119b69c08 RBX: ffff880119b69be8 RCX: ffff8800283c0000
Nov 6 04:07:01 vstest-01 kernel: RDX: ffff88012dbd2000 RSI: ffff8800283df3a0 RDI: ffff88012d95ae40
Nov 6 04:07:01 vstest-01 kernel: RBP: ffffffff8100be4e R08: ffff880119b68000 R09: 0000000000000000
Nov 6 04:07:01 vstest-01 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff880119b69c98
Nov 6 04:07:01 vstest-01 kernel: R13: 0001e66dcaf6492a R14: ffffffff8100c11b R15: ffff880119b69c38
Nov 6 04:07:01 vstest-01 kernel: FS: 000000004277d950(0063) GS:ffff8800283c0000(0000) knlGS:0000000000000000
Nov 6 04:07:01 vstest-01 kernel: CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
Nov 6 04:07:01 vstest-01 kernel: CR2: 00000000006b97f8 CR3: 0000000112f74000 CR4: 00000000000006e0
Nov 6 04:07:01 vstest-01 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 6 04:07:01 vstest-01 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Nov 6 04:07:01 vstest-01 kernel: Call Trace:
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa05a65f9>] ? vmx_vcpu_load+0x99/0x140 [kvm_intel]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa0551a49>] ? kvm_arch_vcpu_load+0x29/0x110 [kvm]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa054861f>] ? vcpu_load+0x3f/0x50 [kvm]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa05486ac>] ? kvm_vcpu_block+0x7c/0xc0 [kvm]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff81093450>] ? autoremove_wake_function+0x0/0x40
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa055a3b9>] ? kvm_arch_vcpu_ioctl_run+0x3f9/0xf70 [kvm]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffffa0544493>] ? kvm_vcpu_ioctl+0x433/0x790 [kvm]
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8111e06a>] ? fire_user_return_notifiers+0x3a/0x50
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8119a1f6>] ? vfs_ioctl+0x36/0xb0
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8119a6c8>] ? do_vfs_ioctl+0x3b8/0x5e0
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8119a6f5>] ? do_vfs_ioctl+0x3e5/0x5e0
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff810b1b89>] ? sys_futex+0x89/0x160
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff81187da1>] ? fget_light+0x71/0xc0
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8119a93f>] ? sys_ioctl+0x4f/0x80
Nov 6 04:07:01 vstest-01 kernel: [<ffffffff8100b302>] ? system_call_fastpath+0x16/0x1b
Nov 6 04:08:25 vstest-01 kernel: BUG: soft lockup - CPU#0 stuck for 67s! [kvm:67059]
Nov 6 04:08:25 vstest-01 kernel: Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm simfs nf_conntrack(-) nf_defrag_ipv4 nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota vzdev ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit xt_dscp ipt_REJECT ip_tables vzevent ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi bridge stp llc snd_pcsp snd_pcm snd_timer snd soundcore dcdbas snd_page_alloc tpm_tis tpm serio_raw tpm_bios ghes i5000_edac edac_core i5k_amb hwmon hed shpchp ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot sg floppy ata_piix ata_generic pata_acpi bnx2 megaraid_sas tg3 [last unloaded: nf_conntrack_ipv4]

If I manually stop the KVM VE before hitting restart, the restart goes well.

 

[SuSt]

Seems to be a problem during stop process with KVM.

If I manually stop the KVM VE before hitting restart, the restart goes well.

As of me, I could not find a direct relationship between KVM running and server fault on reboot. I always manually stop all KVM virtual machines before performing any operations.

Usually after message "Will now restart" in console I see some strings about "shutdown md devices". And this time, the halt process has not reached them. So, I suppose that there may be some bugs in mdadm or SATA/SCSI implementations.

Inside VirtualBox environment everything works fine. To test on bare metal is quite difficult for me, because I have not so much spare hardware.