serios problems with lxc containers

[cryptolukas]

Hi guys,

I made a stop and and start command on a lxc container.
After then, my syslog show this output.
I don't understand it. This problem isn't nice for production.

Apr 27 16:15:18 pve02 kernel: [222994.269773] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:15:45 pve02 systemd[1]: lxc@104.service start operation timed out. Terminating.
Apr 27 16:15:45 pve02 systemd[1]: lxc@104.service: control process exited, code=killed status=30
Apr 27 16:15:45 pve02 systemd[1]: Failed to start LXC Container: 104.
Apr 27 16:15:45 pve02 systemd[1]: Unit lxc@104.service entered failed state.
Apr 27 16:15:45 pve02 pvedaemon[2558]: command 'systemctl start lxc@104' failed: exit code 1
Apr 27 16:15:45 pve02 pvedaemon[4054]: <root@pam> end task UPID:pve02:000009FE:015431EB:5901FCB6:vzstart:104:root@pam: command 'systemctl start lxc@104' failed: exit code 1

Message from syslogd@pve02 at Apr 27 16:15:49 ...
 kernel:[223024.415061] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:15:49 pve02 kernel: [223024.415061] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:16:03 pve02 pveproxy[9987]: proxy detected vanished client connection
Apr 27 16:16:06 pve02 charon: 05[NET] received packet: from 185.17.207.215[64669] to 88.198.230.2[4500] (92 bytes)
Apr 27 16:16:06 pve02 charon: 05[ENC] parsed INFORMATIONAL request 6 [ ]
Apr 27 16:16:06 pve02 charon: 05[ENC] generating INFORMATIONAL response 6 [ ]
Apr 27 16:16:06 pve02 charon: 05[NET] sending packet: from 88.198.230.2[4500] to 185.17.207.215[64669] (76 bytes)
Apr 27 16:16:24 pve02 kernel: [223060.236069] INFO: task lxc-start:2565 blocked for more than 120 seconds.
Apr 27 16:16:24 pve02 kernel: [223060.236096]       Tainted: G           O    4.4.49-1-pve #1
Apr 27 16:16:24 pve02 kernel: [223060.236118] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 27 16:16:24 pve02 kernel: [223060.236157] lxc-start       D ffff880d67b8fcb8     0  2565      1 0x00000000
Apr 27 16:16:24 pve02 kernel: [223060.236185]  ffff880d67b8fcb8 ffffffff821f7d80 ffff880fe70ccb00 ffff880e353e8000
Apr 27 16:16:24 pve02 kernel: [223060.236225]  ffff880d67b90000 ffffffff81f092e4 ffff880e353e8000 00000000ffffffff
Apr 27 16:16:24 pve02 kernel: [223060.236311]  ffffffff81f092e8 ffff880d67b8fcd0 ffffffff8185c215 ffffffff81f092e0
Apr 27 16:16:24 pve02 kernel: [223060.236365] Call Trace:
Apr 27 16:16:24 pve02 kernel: [223060.236386]  [<ffffffff8185c215>] schedule+0x35/0x80
Apr 27 16:16:24 pve02 kernel: [223060.236421]  [<ffffffff8185c4ce>] schedule_preempt_disabled+0xe/0x10
Apr 27 16:16:24 pve02 kernel: [223060.236446]  [<ffffffff8185e1c9>] __mutex_lock_slowpath+0xb9/0x130
Apr 27 16:16:24 pve02 kernel: [223060.236471]  [<ffffffff8185e25f>] mutex_lock+0x1f/0x30
Apr 27 16:16:24 pve02 kernel: [223060.236494]  [<ffffffff81734c0e>] copy_net_ns+0x6e/0x120
Apr 27 16:16:24 pve02 kernel: [223060.236518]  [<ffffffff810a1fbb>] create_new_namespaces+0x11b/0x1e0
Apr 27 16:16:24 pve02 kernel: [223060.236542]  [<ffffffff810a20f3>] copy_namespaces+0x73/0xa0
Apr 27 16:16:24 pve02 kernel: [223060.236565]  [<ffffffff8107f79b>] copy_process+0x8db/0x1c00
Apr 27 16:16:24 pve02 kernel: [223060.236589]  [<ffffffff81080c50>] _do_fork+0x80/0x360
Apr 27 16:16:24 pve02 kernel: [223060.236611]  [<ffffffff81003356>] ? exit_to_usermode_loop+0xa6/0xd0
Apr 27 16:16:24 pve02 kernel: [223060.236636]  [<ffffffff81080fd9>] SyS_clone+0x19/0x20
Apr 27 16:16:24 pve02 kernel: [223060.236658]  [<ffffffff81860336>] entry_SYSCALL_64_fastpath+0x16/0x75
Apr 27 16:16:29 pve02 postfix/smtpd[3161]: connect from unknown[189.207.48.74]

Message from syslogd@pve02 at Apr 27 16:16:29 ...
 kernel:[223064.827556] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:16:29 pve02 kernel: [223064.827556] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:16:30 pve02 postfix/smtpd[3161]: disconnect from unknown[189.207.48.74]

Message from syslogd@pve02 at Apr 27 16:16:39 ...
 kernel:[223074.878727] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:16:39 pve02 kernel: [223074.878727] unregister_netdevice: waiting for lo to become free. Usage count = 1
Apr 27 16:16:42 pve02 pveproxy[9987]: proxy detected vanished client connection
Apr 27 16:16:45 pve02 pveproxy[9987]: proxy detected vanished client connection
Apr 27 16:16:51 pve02 pveproxy[9987]: proxy detected vanished client connection
Apr 27 16:16:51 pve02 pveproxy[9987]: proxy detected vanished client connection

 

[fabian]

please post the full journal from before stopping the container to after starting it again. also include output of "pveversion -v" and "pct config 104". thanks!

 

[grin]

https://github.com/lxc/lxc/issues/2141
Namespace clone results in hang: uninterruptible sleep #2141
Seems to be fixed in kernel 4.14.16 - 4.14.20