Memory cgroup out of memory

StormLXC

Member
Sep 23, 2020
7
1
6
36
We have a Proxmox cluster with several nodes.

All nodes identical: 64GB RAM, 48 Xeon threads, 1x960ssd, ext4 noatime, vm.swappiness=1. Lately, swap was disabled via 'swapoff -a'

4 identical containers running same parsing-task: 20cpu, 16GB, noatime, Ubuntu 20.10 downloaded via ProxMox GUI.

Journaling is disabled at node and inside containers with:

Code:
echo "Storage=none" >> /etc/systemd/journald.conf
service systemd-journald restart

Nodes are hit by OOM-killer every 15 min or so.


Node's GUI shows less than 50% RAM used. Monitoring node with 'top' showed that oom comes when free memory == 0 and almost all free ram is consumed by kernel and allocated in 'buff/cache'. As far as I understand, kernel should release 'buff/cache' once there is memory request from applications, but in this case it probably doesn't.

I tried to overcommit RAM in LXC from 16Gb to 32Gb and it helped a lot. Now, oom visits only 5% of nodes but the problem is not 100% solved.
 
Last edited:
Here is a crash log:

Code:
Apr 11 16:20:48 foxtrot14 kernel: lxcfs invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=1, oom_score_adj=0
Apr 11 16:20:48 foxtrot14 kernel: CPU: 25 PID: 57322 Comm: lxcfs Tainted: P           O      5.4.106-1-pve #1
Apr 11 16:20:48 foxtrot14 kernel: Call Trace:
Apr 11 16:20:48 foxtrot14 kernel:  dump_stack+0x6d/0x8b
Apr 11 16:20:48 foxtrot14 kernel:  dump_header+0x4f/0x1e1
Apr 11 16:20:48 foxtrot14 kernel:  oom_kill_process.cold.33+0xb/0x10
Apr 11 16:20:48 foxtrot14 kernel:  out_of_memory+0x1ad/0x490
Apr 11 16:20:48 foxtrot14 kernel:  __alloc_pages_slowpath+0xd40/0xe30
Apr 11 16:20:48 foxtrot14 kernel:  ? __switch_to_asm+0x34/0x70
Apr 11 16:20:48 foxtrot14 kernel:  ? __switch_to_asm+0x40/0x70
Apr 11 16:20:48 foxtrot14 kernel:  __alloc_pages_nodemask+0x2df/0x330
Apr 11 16:20:48 foxtrot14 kernel:  alloc_pages_current+0x81/0xe0
Apr 11 16:20:48 foxtrot14 kernel:  __get_free_pages+0x11/0x40
Apr 11 16:20:48 foxtrot14 kernel:  pgd_alloc+0x36/0x1e0
Apr 11 16:20:48 foxtrot14 kernel:  mm_init+0x195/0x280
Apr 11 16:20:48 foxtrot14 kernel:  dup_mm+0x68/0x5c0
Apr 11 16:20:48 foxtrot14 kernel:  ? __lock_task_sighand+0x28/0x70
Apr 11 16:20:48 foxtrot14 kernel:  copy_process+0x18a9/0x1b60
Apr 11 16:20:48 foxtrot14 kernel:  _do_fork+0x85/0x350
Apr 11 16:20:48 foxtrot14 kernel:  __x64_sys_clone+0x8f/0xb0
Apr 11 16:20:48 foxtrot14 kernel:  do_syscall_64+0x57/0x190
Apr 11 16:20:48 foxtrot14 kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 11 16:20:48 foxtrot14 kernel: RIP: 0033:0x7f56bbad47be
Apr 11 16:20:48 foxtrot14 kernel: Code: db 0f 85 25 01 00 00 64 4c 8b 0c 25 10 00 00 00 45 31 c0 4d 8d 91 d0 02 00 00 31 d2 31 f6 bf 11 00 20 01 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 b6 00 00 00 41 89 c4 85 c0 0f 85 c3 00 00
Apr 11 16:20:48 foxtrot14 kernel: RSP: 002b:00007f56ba9dc630 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
Apr 11 16:20:48 foxtrot14 kernel: RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f56bbad47be
Apr 11 16:20:48 foxtrot14 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
Apr 11 16:20:48 foxtrot14 kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 00007f56ba9dd700
Apr 11 16:20:48 foxtrot14 kernel: R10: 00007f56ba9dd9d0 R11: 0000000000000246 R12: 000000000000af6b
Apr 11 16:20:48 foxtrot14 kernel: R13: 00007f56ba9dc720 R14: ffffffffffffffff R15: 00007f56bb9f2fe0
Apr 11 16:20:48 foxtrot14 kernel: Mem-Info:
Apr 11 16:20:48 foxtrot14 kernel: active_anon:7661538 inactive_anon:7598803 isolated_anon:0
 active_file:1855 inactive_file:4790 isolated_file:24
 unevictable:48805 dirty:42 writeback:50 unstable:0
 slab_reclaimable:262839 slab_unreclaimable:363214
 mapped:7425017 shmem:7605001 pagetables:212672 bounce:0
 free:55528 free_pcp:1 free_cma:0
Apr 11 16:20:48 foxtrot14 kernel: Node 0 active_anon:14339500kB inactive_anon:16012344kB active_file:2716kB inactive_file:8140kB unevictable:185264kB isolated(anon):0kB isolated(file):0kB mapped:15693904kB dirty:8kB writeback:128kB shmem:16036684kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Apr 11 16:20:48 foxtrot14 kernel: Node 1 active_anon:16306652kB inactive_anon:14382868kB active_file:4704kB inactive_file:11020kB unevictable:9956kB isolated(anon):0kB isolated(file):96kB mapped:14006164kB dirty:160kB writeback:72kB shmem:14383320kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Apr 11 16:20:48 foxtrot14 kernel: Node 0 DMA free:15900kB min:20kB low:32kB high:44kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15984kB managed:15900kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Apr 11 16:20:48 foxtrot14 kernel: lowmem_reserve[]: 0 2910 32055 32055 32055
Apr 11 16:20:48 foxtrot14 kernel: Node 0 DMA32 free:120416kB min:4076kB low:7056kB high:10036kB active_anon:990784kB inactive_anon:1515580kB active_file:120kB inactive_file:92kB unevictable:0kB writepending:8kB present:3094192kB managed:3028656kB mlocked:0kB kernel_stack:15876kB pagetables:62436kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Apr 11 16:20:48 foxtrot14 kernel: lowmem_reserve[]: 0 0 29144 29144 29144
Apr 11 16:20:48 foxtrot14 kernel: Node 0 Normal free:40668kB min:40832kB low:70676kB high:100520kB active_anon:13348716kB inactive_anon:14496764kB active_file:2512kB inactive_file:8292kB unevictable:185264kB writepending:128kB present:30408704kB managed:29852304kB mlocked:185264kB kernel_stack:57200kB pagetables:287864kB bounce:0kB free_pcp:4kB local_pcp:0kB free_cma:0kB
Apr 11 16:20:48 foxtrot14 kernel: lowmem_reserve[]: 0 0 0 0 0
Apr 11 16:20:48 foxtrot14 kernel: Node 1 Normal free:45128kB min:45176kB low:78196kB high:111216kB active_anon:16306652kB inactive_anon:14382868kB active_file:4780kB inactive_file:11280kB unevictable:9956kB writepending:232kB present:33554428kB managed:33021948kB mlocked:9956kB kernel_stack:89804kB pagetables:500388kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Apr 11 16:20:48 foxtrot14 kernel: lowmem_reserve[]: 0 0 0 0 0
Apr 11 16:20:48 foxtrot14 kernel: Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15900kB
Apr 11 16:20:48 foxtrot14 kernel: Node 0 DMA32: 554*4kB (UMEH) 4535*8kB (UM) 3229*16kB (UME) 897*32kB (UMEH) 6*64kB (MH) 4*128kB (H) 2*256kB (H) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 120272kB
Apr 11 16:20:48 foxtrot14 kernel: Node 0 Normal: 10310*4kB (UMH) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 41240kB
Apr 11 16:20:48 foxtrot14 kernel: Node 1 Normal: 11307*4kB (UMH) 2*8kB (H) 2*16kB (H) 3*32kB (H) 2*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 46396kB
Apr 11 16:20:48 foxtrot14 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Apr 11 16:20:48 foxtrot14 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Apr 11 16:20:48 foxtrot14 kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Apr 11 16:20:48 foxtrot14 kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Apr 11 16:20:48 foxtrot14 kernel: 7615504 total pagecache pages
Apr 11 16:20:48 foxtrot14 kernel: 0 pages in swap cache
Apr 11 16:20:48 foxtrot14 kernel: Swap cache stats: add 0, delete 0, find 0/0
Apr 11 16:20:48 foxtrot14 kernel: Free swap  = 0kB
Apr 11 16:20:48 foxtrot14 kernel: Total swap = 0kB
Apr 11 16:20:48 foxtrot14 kernel: 16768327 pages RAM
Apr 11 16:20:48 foxtrot14 kernel: 0 pages HighMem/MovableOnly
Apr 11 16:20:48 foxtrot14 kernel: 288625 pages reserved
Apr 11 16:20:48 foxtrot14 kernel: 0 pages cma reserved
Apr 11 16:20:48 foxtrot14 kernel: 0 pages hwpoisoned
Apr 11 16:20:48 foxtrot14 kernel: Tasks state (memory values in pages):
Apr 11 16:20:48 foxtrot14 kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Apr 11 16:20:48 foxtrot14 kernel: [    765]     0   765     9970     1647    94208        0             0 systemd-journal

......... cut, too long ........

Apr 11 16:20:48 foxtrot14 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/lxc/130/ns/system.slice/pm2-root.service,task=chrome,pid=51370,uid=100000
Apr 11 16:20:48 foxtrot14 kernel: Out of memory: Killed process 51370 (chrome) total-vm:38595940kB, anon-rss:194704kB, file-rss:0kB, shmem-rss:317824kB, UID:100000 pgtables:2092kB oom_score_adj:300
Apr 11 16:20:48 foxtrot14 kernel: oom_reaper: reaped process 51370 (chrome), now anon-rss:0kB, file-rss:0kB, shmem-rss:318384kB
 
egrep -e oom-killer /var/log/kern.log

Code:
Apr 11 03:02:13 foxtrot14 kernel: [706780.765069] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 03:18:36 foxtrot14 kernel: [707763.668747] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 03:19:14 foxtrot14 kernel: [707801.545968] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 03:22:42 foxtrot14 kernel: [708009.259695] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 04:27:53 foxtrot14 kernel: [711920.612637] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 05:22:01 foxtrot14 kernel: [715168.814644] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 05:30:56 foxtrot14 kernel: [715703.067057] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 05:34:08 foxtrot14 kernel: [715895.678606] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 06:35:47 foxtrot14 kernel: [719594.137018] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 06:43:26 foxtrot14 kernel: [720053.824058] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 06:43:50 foxtrot14 kernel: [720077.000294] ThreadPoolServi invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 06:57:09 foxtrot14 kernel: [720876.058310] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 06:57:27 foxtrot14 kernel: [720894.793756] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 07:46:18 foxtrot14 kernel: [723825.824249] server invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 08:32:53 foxtrot14 kernel: [726620.717799] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 08:34:17 foxtrot14 kernel: [726704.674956] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 09:00:36 foxtrot14 kernel: [728283.711599] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 09:00:48 foxtrot14 kernel: [728295.183814] server invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 09:01:06 foxtrot14 kernel: [728313.647071] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 09:15:02 foxtrot14 kernel: [729149.551796] VizCompositorTh invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=200
Apr 11 09:15:38 foxtrot14 kernel: [729184.997177] ThreadPoolForeg invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 09:16:22 foxtrot14 kernel: [729229.401372] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 09:41:54 foxtrot14 kernel: [730761.242164] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 09:42:05 foxtrot14 kernel: [730772.294139] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 09:42:19 foxtrot14 kernel: [730786.637332] ThreadPoolForeg invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 11:12:10 foxtrot14 kernel: [736177.561916] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 11:19:48 foxtrot14 kernel: [736635.223469] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 11:41:23 foxtrot14 kernel: [737930.325193] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:05:59 foxtrot14 kernel: [739406.798018] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:10:11 foxtrot14 kernel: [739658.784261] server invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Apr 11 12:10:18 foxtrot14 kernel: [739665.311359] ThreadPoolServi invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:13:25 foxtrot14 kernel: [739852.602297] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 12:13:52 foxtrot14 kernel: [739879.446842] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 12:16:27 foxtrot14 kernel: [740034.729350] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:16:36 foxtrot14 kernel: [740043.598843] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:16:42 foxtrot14 kernel: [740049.587784] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:16:51 foxtrot14 kernel: [740058.200034] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:23:12 foxtrot14 kernel: [740439.128975] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 12:28:17 foxtrot14 kernel: [740744.720847] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:29:14 foxtrot14 kernel: [740801.527825] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 12:29:27 foxtrot14 kernel: [740814.034323] server invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Apr 11 12:29:43 foxtrot14 kernel: [740829.944858] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 12:59:31 foxtrot14 kernel: [742618.012701] ThreadPoolForeg invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:59:31 foxtrot14 kernel: [742618.079794] ThreadPoolForeg invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 12:59:48 foxtrot14 kernel: [742635.083829] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 13:06:33 foxtrot14 kernel: [743039.989527] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 13:08:02 foxtrot14 kernel: [743129.432127] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 13:15:49 foxtrot14 kernel: [743596.179733] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 13:16:00 foxtrot14 kernel: [743607.282817] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 13:19:34 foxtrot14 kernel: [743821.315836] server invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 13:27:03 foxtrot14 kernel: [744270.488143] Chrome_IOThread invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 13:36:49 foxtrot14 kernel: [744856.000214] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 13:55:08 foxtrot14 kernel: [745954.871300] ThreadPoolForeg invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 13:56:03 foxtrot14 kernel: [746010.188593] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 13:56:08 foxtrot14 kernel: [746014.912290] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 14:02:12 foxtrot14 kernel: [746379.312492] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 14:32:25 foxtrot14 kernel: [748192.766822] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 14:35:54 foxtrot14 kernel: [748401.617084] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 14:38:49 foxtrot14 kernel: [748576.433820] NetworkService invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 14:39:25 foxtrot14 kernel: [748612.495395] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 14:57:15 foxtrot14 kernel: [749682.485921] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=300
Apr 11 14:57:19 foxtrot14 kernel: [749686.492002] chrome invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 11 15:16:20 foxtrot14 kernel: [750827.772208] server invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
Apr 11 16:20:48 foxtrot14 kernel: [754695.717616] lxcfs invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=1, oom_score_adj=0
Apr 11 21:16:45 foxtrot14 kernel: [772451.963339] server invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
 
Did you find a solution or explanation for this? I have a cluster of 3 identical nodes. The container which terminates without warning is a debian based openldap server. Small resource consumer.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!