I just updated the bugreport https://bugzilla.proxmox.com/show_bug.cgi?id=1943 - sadly could still not reproduce the issue locally (despite sending out quite some (fragmented) ipv6 traffic and restaring containers). If possible please provide the requested information in the bug-report Thanks!
I noted this on the bug as well.. We ( @seneca214 and me) were able to reproduce the bug with the ip6tables block in place, unfortunately. This time the spinlock was in a kernel tree with the ipv4 version of the same exit_frags_net kernel process. @seneca214 noted that there was a lot of mDNS broadcast traffic hitting this machine so maybe that's what is doing it. I enabled the firewall at the cluster level and added MDNS macro to the drop chain and then made sure the default action was to allow so that we didn't lose access to anything else while testing this.
+1 me too. I am not using the proxmox fw at all (disabled) and up to this point I have not seen this behavior before. Some nodes work fine, some are seeing this issue. pveversion for all nodes: Code: proxmox-ve: 5.3-1 (running kernel: 4.15.18-11-pve) pve-manager: 5.3-9 (running version: 5.3-9/ba817b29) pve-kernel-4.15: 5.3-2 pve-kernel-4.15.18-11-pve: 4.15.18-33 pve-kernel-4.15.18-9-pve: 4.15.18-30 pve-kernel-4.15.18-7-pve: 4.15.18-27 ceph: 12.2.11-pve1 corosync: 2.4.4-pve1 criu: 2.11.1-1~bpo90 glusterfs-client: 3.8.8-1 ksm-control-daemon: 1.2-2 libjs-extjs: 6.0.1-2 libpve-access-control: 5.1-3 libpve-apiclient-perl: 2.0-5 libpve-common-perl: 5.0-46 libpve-guest-common-perl: 2.0-20 libpve-http-server-perl: 2.0-11 libpve-storage-perl: 5.0-38 libqb0: 1.0.3-1~bpo9 lvm2: 2.02.168-pve6 lxc-pve: 3.1.0-3 lxcfs: 3.0.3-pve1 novnc-pve: 1.0.0-2 openvswitch-switch: 2.7.0-3 proxmox-widget-toolkit: 1.0-22 pve-cluster: 5.0-33 pve-container: 2.0-34 pve-docs: 5.3-2 pve-edk2-firmware: 1.20181023-1 pve-firewall: 3.0-17 pve-firmware: 2.0-6 pve-ha-manager: 2.0-6 pve-i18n: 1.0-9 pve-libspice-server1: 0.14.1-2 pve-qemu-kvm: 2.12.1-1 pve-xtermjs: 3.10.1-1 qemu-server: 5.0-46 smartmontools: 6.5+svn4324-1 spiceterm: 3.0-5 vncterm: 1.5-3 zfsutils-linux: 0.7.12-pve1~bpo1
@alexskysilk : * Please provide the perf-data, workqueue trace and other information as requested in: https://bugzilla.proxmox.com/show_bug.cgi?id=1943#c4 I just updated the issue's summary and added a comment to clarify what the exact problem described in the issue is (kworker spinning in inet_frags_exit_net) - given that we had quite a few reports of other issues with the same symptoms (kworker using 100% CPU - only fixable by node reset)
Does your workaround with iptables still work and prevent the issue from occuring (for those users, which tried to mitigate the issue with it)? As written in the bugreport (https://bugzilla.proxmox.com/show_bug.cgi?id=1943#c20) I still was not able to reproduce the issue locally despite additionally introducing mDNS traffic into the test-setup
So far, we've been unable to reproduce the issue with any server that's been rebooted with the firewall rules in place. If nothing else, this seems to greatly mitigate the issue.
Will do, but wont be able to get to it till next week. I'll keep an eye on bugzilla for any further developments.
