lxc crashed

[radekk]

Hello,

I have a random lxc crash for no (know to me) reason. The only info I can get in the logs:

Aug 25 19:48:54 router kernel: [3554506.426835] CPU: 4 PID: 49264 Comm: systemd-journal Tainted: P           O      5.4.124-1-pve #1
Aug 25 19:48:54 router kernel: [3554506.426891]  mem_cgroup_out_of_memory+0xc4/0xd0
Aug 25 19:48:54 router kernel: [3554506.426918]  shmem_fallocate+0x339/0x4b0
Aug 25 19:48:54 router kernel: [3554506.426949]  do_syscall_64+0x57/0x190
Aug 25 19:48:54 router kernel: [3554506.426981] RDX: 0000000001000000 RSI: 0000000000000000 RDI: 0000000000000013
Aug 25 19:48:54 router kernel: [3554506.427032] kmem: usage 46376kB, limit 9007199254740988kB, failcnt 0
Aug 25 19:48:54 router kernel: [3554506.427071] [   3127]     0  3127      604       28    40960        0             0 agetty
Aug 25 19:48:54 router kernel: [3554506.427094] [   2958]     0  2958     1377       53    49152        0             0 cron
Aug 25 19:48:54 router kernel: [3554506.427137] [   2746]     0  2746    42609      592    98304        0             0 systemd

My PVE is:

# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.124-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.119-1-pve: 5.4.119-1
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve4~bpo10
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
openvswitch-switch: 2.12.3-1
proxmox-backup-client: 1.1.12-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.5-pve1~bpo10+1

I'm looking for any hints how to find the root cause of the crash.

Radek

 

[oguz]

hi,

you've done kernel upgrades on host but didn't reboot:

# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.124-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.124-1-pve: 5.4.124-2

please do that, and if the crash happens again post the container configuration (pct config CTID)

 

[radekk]

hi,

you've done kernel upgrades on host but didn't reboot:

# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.124-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.124-1-pve: 5.4.124-2

please do that, and if the crash happens again post the container configuration (pct config CTID)

It is very plausible. I'll do that ASAP. Thanks for the hints.

Radek