Errors in kern.log

[odesport]

Hi,

A lot of errors are written in kern.log, when VMs are launched :

 WARNING: at net/core/dev.c:1711 skb_gso_segment+0x220/0x310() (Tainted: G        W  ---------------   )
Jul 17 15:36:38 aragon3 kernel: Hardware name: PowerEdge M620
Jul 17 15:36:38 aragon3 kernel: tun: caps=(0x80000000, 0x0) len=9827 data_len=8327 ip_summed=1
Jul 17 15:36:38 aragon3 kernel: Modules linked in: ext4 jbd2 vzethdev vznetdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 vzcpt nf_conntrack vzdquota vzmon vzdev ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit xt_dscp vhost_net ipt_REJECT tun macvtap macvlan kvm_intel ip_tables kvm dlm configfs fuse vzevent dm_round_robin ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nfsd nfs auth_rpcgss nfs_acl fscache lockd sunrpc bonding 8021q garp ipv6 xfs dm_multipath snd_pcsp snd_pcm snd_page_alloc snd_timer snd soundcore sb_edac dcdbas iTCO_wdt wmi edac_core iTCO_vendor_support shpchp power_meter ext3 mbcache jbd ses sg enclosure ahci megaraid_sas bnx2x mdio [last unloaded: scsi_wait_scan]
Jul 17 15:36:38 aragon3 kernel: Pid: 3845, comm: rsyslogd veid: 0 Tainted: G        W  ---------------    2.6.32-20-pve #1
Jul 17 15:36:38 aragon3 kernel: Call Trace:
Jul 17 15:36:38 aragon3 kernel: <IRQ>  [<ffffffff8106b3e7>] ? warn_slowpath_common+0x87/0xe0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8106b4f6>] ? warn_slowpath_fmt+0x46/0x50
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814494a0>] ? skb_gso_segment+0x220/0x310
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144bc42>] ? dev_hard_start_xmit+0x1b2/0x4f0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81480ae6>] ? ip_local_deliver_finish+0x126/0x2e0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81468e7a>] ? sch_direct_xmit+0x16a/0x1d0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144c445>] ? dev_queue_xmit+0x4c5/0x6a0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e2b80>] ? br_dev_queue_push_xmit+0x60/0xc0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e2c38>] ? br_forward_finish+0x58/0x60
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e2f2b>] ? __br_forward+0x9b/0xc0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8147660c>] ? nf_hook_slow+0xac/0x120
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e3e20>] ? br_handle_frame_finish+0x0/0x2f0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e30bd>] ? br_forward+0x5d/0x70
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e4017>] ? br_handle_frame_finish+0x1f7/0x2f0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814e42ba>] ? br_handle_frame+0x1aa/0x250
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144c91c>] ? __netif_receive_skb+0x23c/0x6e0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144d008>] ? napi_skb_finish+0x38/0x70
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144cf08>] ? netif_receive_skb+0x58/0x60
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8143de85>] ? __alloc_skb+0x95/0x260
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144d020>] ? napi_skb_finish+0x50/0x70
Jul 17 15:36:38 aragon3 kernel: [<ffffffff814f24ac>] ? vlan_gro_receive+0x8c/0xa0
Jul 17 15:36:38 aragon3 kernel: [<ffffffffa004197a>] ? bnx2x_rx_int+0xefa/0x16b0 [bnx2x]
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81051626>] ? enqueue_task+0x66/0x80
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81010fef>] ? ich_force_enable_hpet+0x1cf/0x200
Jul 17 15:36:38 aragon3 kernel: [<ffffffffa00421d4>] ? bnx2x_poll+0xa4/0x2e0 [bnx2x]
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8144d879>] ? net_rx_action+0x199/0x3c0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81074c7b>] ? __do_softirq+0x11b/0x260
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8100c32c>] ? call_softirq+0x1c/0x30
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8100de95>] ? do_softirq+0x75/0xb0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81074f55>] ? irq_exit+0xc5/0xd0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81523452>] ? do_IRQ+0x72/0xe0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8100bb13>] ? ret_from_intr+0x0/0x11
Jul 17 15:36:38 aragon3 kernel: <EOI>  [<ffffffff8106c3b8>] ? do_syslog+0x218/0x670
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8106c3e5>] ? do_syslog+0x245/0x670
Jul 17 15:36:38 aragon3 kernel: [<ffffffff810b4a59>] ? do_futex+0x159/0xad0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81208100>] ? kmsg_read+0x0/0x60
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81095300>] ? autoremove_wake_function+0x0/0x40
Jul 17 15:36:38 aragon3 kernel: [<ffffffff81208143>] ? kmsg_read+0x43/0x60
Jul 17 15:36:38 aragon3 kernel: [<ffffffff811fc5db>] ? proc_reg_read+0x7b/0xc0
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8151ad3c>] ? thread_return+0xbc/0x870
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8119333e>] ? vfs_read+0x9e/0x190
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8119347a>] ? sys_read+0x4a/0x90
Jul 17 15:36:38 aragon3 kernel: [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
Jul 17 15:36:38 aragon3 kernel: ---[ end trace ba10d782271997b7 ]---
Jul 17 15:40:23 aragon3 kernel: vmbr0: port 2(tap101i0) entering disabled state
Jul 17 15:40:23 aragon3 kernel: vmbr0: port 2(tap101i0) entering disabled state
Jul 17 15:40:23 aragon3 kernel: vmbr1: port 2(tap101i1) entering disabled state
Jul 17 15:40:23 aragon3 kernel: vmbr1: port 2(tap101i1) entering disabled state
Jul 17 15:40:39 aragon3 kernel: device tap101i0 entered promiscuous mode
Jul 17 15:40:39 aragon3 kernel: vmbr0: port 2(tap101i0) entering forwarding state
Jul 17 15:40:39 aragon3 kernel: device tap101i1 entered promiscuous mode
Jul 17 15:40:39 aragon3 kernel: vmbr1: port 2(tap101i1) entering forwarding state
Jul 17 15:40:50 aragon3 kernel: tap101i0: no IPv6 routers present
Jul 17 15:40:50 aragon3 kernel: tap101i1: no IPv6 routers present
Jul 17 15:45:13 aragon3 kernel: kvm: 226574: cpu0 unhandled rdmsr: 0x345
Jul 17 15:45:13 aragon3 kernel: kvm: 226574: cpu0 unhandled wrmsr: 0x40 data 0
Jul 17 15:45:13 aragon3 kernel: kvm: 226574: cpu0 unhandled wrmsr: 0x60 data 0

I've tried different processors (kvm64, qemu64, core2duo), network cards (virio, intel) or hard disks (virtio, scsi) but there's no change.

pveversion --verbose
pve-manager: 3.0-23 (pve-manager/3.0/957f0862)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-100
pve-kernel-2.6.32-20-pve: 2.6.32-100
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-4
qemu-server: 3.0-20
pve-firmware: 1.0-22
libpve-common-perl: 3.0-4
libpve-access-control: 3.0-4
libpve-storage-perl: 3.0-8
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-13
ksm-control-daemon: 1.1-1

Product : Dell PowerEdge M620
Version : PowerEdge M1000e

 

[tom]

pls test with latest kernel (2.6.32-22-107) and report.

 

[odesport]

I've installed pve-kernel-2.6.32-22-pve_2.6.32-107_amd64.deb but the problem is the same.

 

[odesport]

I've installed Proxmox 3 on Poweredge M610 and the messages doesn't appear.

 

[sgromstad]

Have exactly the same problem with PowerEdge M620 and Proxmox 3.0.
Did you find any answer/fix for this error?

 

[odesport]

The only solution I've found is to disable rx-checksumming on the hypervisors :

ethtool -K eth0 rx off

 

[dietmar]

Do you run a FreeBDS guest?

 

[joevsthevolcano]

I'm hopping into this thread because I am experiencing the same problem on a dell t110. I do indeed have several FreeBSD guests:

Sep  8 21:11:08 asgaard kernel: WARNING: at net/core/dev.c:1758 skb_gso_segment+0x220/0x310() (Tainted: G        W  ---------------   )Sep  8 21:11:08 asgaard kernel: Hardware name: PowerEdge T110
Sep  8 21:11:08 asgaard kernel: tg3: caps=(0x9b3, 0x0) len=4396 data_len=0 ip_summed=0
Sep  8 21:11:08 asgaard kernel: Modules linked in: vzethdev vznetdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 vzcpt nf_conntrack vzdquota vzmon vzdev ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit xt_dscp ipt_REJECT ip_tables vhost_net macvtap macvlan kvm_intel kvm dlm configfs drbd fuse tun vzevent ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc ipv6 dummy snd_pcsp dcdbas snd_pcm snd_page_alloc snd_timer snd soundcore power_meter iTCO_wdt iTCO_vendor_support ext3 mbcache jbd raid1 sg tg3 ahci ata_generic pata_acpi [last unloaded: scsi_wait_scan]
Sep  8 21:11:08 asgaard kernel: Pid: 3461, comm: vhost-3448 veid: 0 Tainted: G        W

 

[dietmar]

My last information on that is that the problem is related to the FreeBSD packet filtering code ('pfctl -e' triggers those messages).

 

[joevsthevolcano]

My last information on that is that the problem is related to the FreeBSD packet filtering code ('pfctl -e' triggers those messages).

Thank you. That did seem to be the problem. As soon as switched from PF to another firewall the errors went away.