unregister_netdevice: waiting for lo to become free

[Kan]

Hello,

I randomly have this message in the console of a container (centos 6.8).

Message from syslogd@vm at Apr 6 10:22:42 ...
kernel:[2401080.372795] unregister_netdevice: waiting for lo to become free. Usage count = 1

The message only exists on container, not in proxmox host (4.4). Even though I migrate CT to another node.

This seems to not affect CT performance (not really sure). But this is not cool for the owner of the CT.

How should I fix this ?

 

[Kan]

No idea ?

 

[David Hooton]

I am seeing this error as well on my host running 5.0-9/c7bdd872

 

[RobFantini]

same here for awhile. reported by a user last week, using latest updates.

 

[fabian]

should be fixed in latest kernels, please include 'pveversion -v' output if you can still reproduce

 

[RobFantini]

Message from syslogd@fbcadmin at Oct 13 07:40:49 ...
 kernel:[414849.455263] unregister_netdevice: waiting for lo to become free. Usage count = 1


# pveversion -v
proxmox-ve: 5.0-23 (running kernel: 4.10.17-3-pve)
pve-manager: 5.0-32 (running version: 5.0-32/2560e073)
pve-kernel-4.10.17-3-pve: 4.10.17-23
libpve-http-server-perl: 2.0-6
lvm2: 2.02.168-pve3
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-12
qemu-server: 5.0-15
pve-firmware: 2.0-2
libpve-common-perl: 5.0-18
libpve-guest-common-perl: 2.0-12
libpve-access-control: 5.0-6
libpve-storage-perl: 5.0-15
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-2
pve-docs: 5.0-9
pve-qemu-kvm: 2.9.1-1
pve-container: 2.0-16
pve-firewall: 3.0-3
pve-ha-manager: 2.0-2
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.0-2
lxcfs: 2.0.7-pve4
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.6.5.11-pve18~bpo90
ceph: 12.2.0-pve1

 

[fabian]

what does uname -a output? there are three kernel versions with ABI 4.10.17-3-pve:

-21: does not yet have the fix
-22: has it
-23: also has it

pveversion does not tell us which of those you are running, but uname -a does. or upgrade to the current -4-pve one, which also has it and requires a reboot anyway

 

[RobFantini]

what does uname -a output? there are three kernel versions with ABI 4.10.17-3-pve:

-21: does not yet have the fix
-22: has it
-23: also has it

pveversion does not tell us which of those you are running, but uname -a does. or upgrade to the current -4-pve one, which also has it and requires a reboot anyway

currently running:

Linux sys3 4.10.17-3-pve #1 SMP PVE 4.10.17-23 (Tue, 19 Sep 2017 09:43:50 +0200) x86_64 GNU/Linux

I rebooted a day ago. let me check when that was installed.. /var/log/apt/history.log:

Start-Date: 2017-10-05  13:38:04
Commandline: apt-get -y dist-upgrade
Upgrade: zfs-initramfs:amd64 (0.6.5.11-pve17~bpo90, 0.6.5.11-pve18~bpo90), zfsutils-linux:amd64 (0.6.5.11-pve17~bpo90, 0.6.5.1
1-pve18~bpo90), libzfs2linux:amd64 (0.6.5.11-pve17~bpo90, 0.6.5.11-pve18~bpo90), libpve-guest-common-perl:amd64 (2.0-11, 2.0-1
2), lxc-pve:amd64 (2.1.0-1, 2.1.0-2), libzpool2linux:amd64 (0.6.5.11-pve17~bpo90, 0.6.5.11-pve18~bpo90), libnvpair1linux:amd64
(0.6.5.11-pve17~bpo90, 0.6.5.11-pve18~bpo90), libuutil1linux:amd64 (0.6.5.11-pve17~bpo90, 0.6.5.11-pve18~bpo90), pve-kernel-4
.10.17-3-pve:amd64 (4.10.17-21, 4.10.17-23), proxmox-ve:amd64 (5.0-21, 5.0-23)
End-Date: 2017-10-05  13:39:12

I checked logs, all 3 nodes in cluster had kernel updated on 10/5 .

I usually reboot all nodes shortly after a kernel upgrade. However there is a chance I did not do so.

 

[RobFantini]

Message from syslogd@dhcp-primary at Oct 17 11:58:19 ...
 kernel:[138340.606861] unregister_netdevice: waiting for lo to become free. Usage count = 1

Message from syslogd@dhcp-primary at Oct 17 11:58:29 ...
 kernel:[138350.687527] unregister_netdevice: waiting for lo to become free. Usage count = 1

that is from a lxc running systemd .

 

[fabian]

seems like there is yet another fix in that area, which is included in 4.12 and thus also in our 4.13.x based kernels which will be available soon in the regular repositories.

 

[RobFantini]

hello - i just want to say i am just reporting not complaining or writing code. if there is anything I can do to help with testing or attaching logs let me know.

*cli within a lxc

Message from syslogd@fbcadmin at Oct 19 16:40:05 ...
 kernel:[23124.404535] unregister_netdevice: waiting for lo to become free. Usage count = 1

lxc hostname 'fbcadmin' does not use systemd.


*host info that lxc runs on.

# pveversion -v
proxmox-ve: 5.0-24 (running kernel: 4.10.17-4-pve)
pve-manager: 5.0-32 (running version: 5.0-32/2560e073)
pve-kernel-4.10.17-4-pve: 4.10.17-24
pve-kernel-4.10.17-3-pve: 4.10.17-23
libpve-http-server-perl: 2.0-6
lvm2: 2.02.168-pve3
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-14
qemu-server: 5.0-15
pve-firmware: 2.0-2
libpve-common-perl: 5.0-18
libpve-guest-common-perl: 2.0-12
libpve-access-control: 5.0-6
libpve-storage-perl: 5.0-15
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-2
pve-docs: 5.0-9
pve-qemu-kvm: 2.9.1-1
pve-container: 2.0-16
pve-firewall: 3.0-3
pve-ha-manager: 2.0-2
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.0-2
lxcfs: 2.0.7-pve4
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.6.5.11-pve18~bpo90
ceph: 12.2.1-pve1

sys3  ~ # dmesg|grep "unregister_netdevice:"
[22546.642672] unregister_netdevice: waiting for lo to become free. Usage count = 1
[23124.404535] unregister_netdevice: waiting for lo to become free. Usage count = 1

should i continue to post these here or at a bug reports?

 

[fabian]

should i continue to post these here or at a bug reports?

if you can still reproduce this once you are running a 4.13 kernel, please bump this thread and I will take another look!

 

[RobFantini]

today i updated and restarted all nodes to run this kernel :

# uname -a 
Linux sys3 4.13.4-1-pve #1 SMP PVE 4.13.4-25 (Fri, 13 Oct 2017 08:59:53 +0200) x86_64 GNU/Linux

 

[bootsie123]

I seem to be starting to get this message now with kernel 4.15.18-17. Any ideas?

 

[Proxygen]

I am seeing this on v6

pve-kernel-helper: 6.0-6
pve-kernel-5.0.18-1-pve: 5.0.18-3
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1
corosync: 3.0.2-pve2
criu: 3.11-3
glusterfs-client: 5.5-3
libjs-extjs: 6.0.1-10
libknet1: 1.10-pve2
libpve-access-control: 6.0-2
libpve-apiclient-perl: 3.0-2
libpve-common-perl: 6.0-3
libpve-guest-common-perl: 3.0-1
libpve-http-server-perl: 3.0-2
libpve-storage-perl: 6.0-7
libqb0: 1.0.5-1
lvm2: 2.03.02-pve3
lxc-pve: 3.1.0-64
lxcfs: 3.0.3-pve60
novnc-pve: 1.0.0-60
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.0-5
pve-cluster: 6.0-5
pve-container: 3.0-5
pve-docs: 6.0-4
pve-edk2-firmware: 2.20190614-1
pve-firewall: 4.0-7
pve-firmware: 3.0-2
pve-ha-manager: 3.0-2
pve-i18n: 2.0-2
pve-qemu-kvm: 4.0.0-5
pve-xtermjs: 3.13.2-1
qemu-server: 6.0-7
smartmontools: 7.0-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.1-pve1

 

[johncsuti]

Is this still a thing? i am having this as well...

pveversion -v

output:

proxmox-ve: 6.3-1 (running kernel: 5.4.106-1-pve)
pve-manager: 6.3-6 (running version: 6.3-6/2184247e)
pve-kernel-5.4: 6.3-8
pve-kernel-helper: 6.3-8
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.0.8
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-5
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.12-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-9
pve-cluster: 6.2-1
pve-container: 3.3-4
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-5
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-10
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1