System freeze after kernel error in latest Proxmox VE 3.3

[lucaferr]

Hello!
I'm getting random freezes (every some hours) of a system running latest Proxmox VE 3.3 with kernel 2.6.32-34-pve. Just before the freeze, this is logged into the syslog. Do you have any idea? I tested the memory with MemTest86 for hours: 100% sane. Thank you very much!

Dec 12 19:58:53 server1 kernel: ------------[ cut here ]------------
Dec 12 19:58:53 server1 kernel: WARNING: at lib/list_debug.c:26 __list_add+0x6c/0x90() (Not tainted)
Dec 12 19:58:53 server1 kernel: Hardware name: MS-7799
Dec 12 19:58:53 server1 kernel: list_add corruption. next->prev should be prev (ffff880036b970d0), but was ffff8800366f70d0. (next=ffff8801195d8260).
Dec 12 19:58:53 server1 kernel: Modules linked in: vzethdev vznetdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt vzdquota vzmon vzdev ip6t_REJECT
 ip6table_mangle ip6table_filter vhost_net ip6_tables tun macvtap macvlan xt_hl kvm_intel kvm xt_TCPMSS vzevent act_police cls_basic cls_flow cls_fw cls_u32 sch_tbf sch
_prio sch_htb sch_hfsc sch_ingress sch_sfq xt_statistic xt_time xt_connlimit xt_realm ip_set_hash_ip iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_
REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype xt_set ip_set nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_ppt
p nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_ud
plite nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack
_h323 nf_conntrack_ftp
Dec 12 19:58:53 server1 kernel: xt_TPROXY nf_defrag_ipv6 nf_tproxy_core xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MA
RK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_AUDIT ipt_LOG xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc ipv6 fuse snd_hda_codec_hdmi i915 iTCO_wdt snd_hda_intel snd_pcsp drm_kms_helper iTCO_vendor_support snd_hda_codec snd_hwdep i2c_i801 drm i2c_algo_bit lpc_ich snd_pcm i2c_core serio_raw wmi mfd_core snd_page_alloc snd_timer snd soundcore video output shpchp ext3 mbcache jbd raid1 sg r8169 mii ahci xhci_hcd [last unloaded: scsi_wait_scan]
Dec 12 19:58:53 server1 kernel: Pid: 425404, comm: vzctl veid: 0 Not tainted 2.6.32-34-pve #1
Dec 12 19:58:53 server1 kernel: Call Trace:
Dec 12 19:58:53 server1 kernel: [<ffffffff810733b7>] ? warn_slowpath_common+0x87/0xe0
Dec 12 19:58:53 server1 kernel: [<ffffffff810734c6>] ? warn_slowpath_fmt+0x46/0x50
Dec 12 19:58:53 server1 kernel: [<ffffffff8129898c>] ? __list_add+0x6c/0x90
Dec 12 19:58:53 server1 kernel: [<ffffffff81156451>] ? vma_prio_tree_add+0xb1/0xd0
Dec 12 19:58:53 server1 kernel: [<ffffffff8107066c>] ? dup_mm+0x4ec/0xad0
Dec 12 19:58:53 server1 kernel: [<ffffffff81071b1f>] ? copy_process.part.25+0xe5f/0x1570
Dec 12 19:58:53 server1 kernel: [<ffffffff81047ad4>] ? __do_page_fault+0x1e4/0x490
Dec 12 19:58:53 server1 kernel: [<ffffffff81072373>] ? do_fork_pid+0x143/0x500
Dec 12 19:58:53 server1 kernel: [<ffffffff811bcb9a>] ? vfs_ioctl+0x2a/0xa0
Dec 12 19:58:53 server1 kernel: [<ffffffff811bd1ce>] ? do_vfs_ioctl+0x7e/0x5a0
Dec 12 19:58:53 server1 kernel: [<ffffffff8107277a>] ? do_fork+0x1a/0x20
Dec 12 19:58:53 server1 kernel: [<ffffffff810099d8>] ? sys_clone+0x28/0x30
Dec 12 19:58:53 server1 kernel: [<ffffffff8100b4a3>] ? stub_clone+0x13/0x20
Dec 12 19:58:53 server1 kernel: [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
Dec 12 19:58:53 server1 kernel: ---[ end trace 9ab24ca951aef0e4 ]---
Dec 12 19:58:53 server1 kernel: Tainting kernel with flag 0x9
Dec 12 19:58:53 server1 kernel: Pid: 425404, comm: vzctl veid: 0 Not tainted 2.6.32-34-pve #1
Dec 12 19:58:53 server1 kernel: Call Trace:
Dec 12 19:58:53 server1 kernel: [<ffffffff81073269>] ? add_taint+0x69/0x70
Dec 12 19:58:53 server1 kernel: [<ffffffff810733d9>] ? warn_slowpath_common+0xa9/0xe0
Dec 12 19:58:53 server1 kernel: [<ffffffff810734c6>] ? warn_slowpath_fmt+0x46/0x50
Dec 12 19:58:53 server1 kernel: [<ffffffff8129898c>] ? __list_add+0x6c/0x90
Dec 12 19:58:53 server1 kernel: [<ffffffff81156451>] ? vma_prio_tree_add+0xb1/0xd0
Dec 12 19:58:53 server1 kernel: [<ffffffff8107066c>] ? dup_mm+0x4ec/0xad0
Dec 12 19:58:53 server1 kernel: [<ffffffff81071b1f>] ? copy_process.part.25+0xe5f/0x1570
Dec 12 19:58:53 server1 kernel: [<ffffffff81047ad4>] ? __do_page_fault+0x1e4/0x490
Dec 12 19:58:53 server1 kernel: [<ffffffff81072373>] ? do_fork_pid+0x143/0x500
Dec 12 19:58:53 server1 kernel: [<ffffffff811bcb9a>] ? vfs_ioctl+0x2a/0xa0
Dec 12 19:58:53 server1 kernel: [<ffffffff811bd1ce>] ? do_vfs_ioctl+0x7e/0x5a0
Dec 12 19:58:53 server1 kernel: [<ffffffff8107277a>] ? do_fork+0x1a/0x20
Dec 12 19:58:53 server1 kernel: [<ffffffff810099d8>] ? sys_clone+0x28/0x30
Dec 12 19:58:53 server1 kernel: [<ffffffff8100b4a3>] ? stub_clone+0x13/0x20
Dec 12 19:58:53 server1 kernel: [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b

 

[e100]

Good results from memtest only mean that if there is a problem memtest86 was unable to detect it.
I have had memtest86 not detect bad RAM numerous times.

But in this case it looks like you may have stumbled across a known kernel bug:
https://bugs.centos.org/view.php?id=6662
https://access.redhat.com/solutions/166583

 

[lucaferr]

Thank you for your answer, e100!
I also have googled a lot finding 2.6 kernel bugs giving similar errors. I don't have any NFS resource mounted on this server (it only uses local storage), so I'd exclude anything related to NFS. Any other ideas?
Thank you very much!