Memory cgroup out of memory : how to understand it ?

E

Ernie95

Member
Sep 1, 2025
75
5
8
Dear All,

For the first time, I have this message :

Code: 
Oct 06 06:20:32 pve kernel: file invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Oct 06 06:20:32 pve kernel: CPU: 9 UID: 100000 PID: 2604997 Comm: file Tainted: P           O        7.0.14-20-pve #1>
Oct 06 06:20:32 pve kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Oct 06 06:20:32 pve kernel: Hardware name: ASUSTeK COMPUTER INC. To be filled by O.E.M./P12R-M Series, BIOS 1901 01/1>
Oct 06 06:20:32 pve kernel: Call Trace:
Oct 06 06:20:32 pve kernel:  <TASK>
Oct 06 06:20:32 pve kernel:  dump_stack_lvl+0x5f/0x90
Oct 06 06:20:32 pve kernel:  dump_stack+0x10/0x18
Oct 06 06:20:32 pve kernel:  dump_header+0x48/0x1c7
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel:  out_of_memory+0x22f/0x4d0
Oct 06 06:20:32 pve kernel:  mem_cgroup_out_of_memory+0x100/0x120
Oct 06 06:20:32 pve kernel:  try_charge_memcg+0x3e6/0x650
Oct 06 06:20:32 pve kernel:  charge_memcg+0x34/0x90
Oct 06 06:20:32 pve kernel:  __mem_cgroup_charge+0x2d/0xa0
Oct 06 06:20:32 pve kernel:  do_anonymous_page+0x379/0x930
Oct 06 06:20:32 pve kernel:  ? __pte_offset_map+0x1c/0x130
Oct 06 06:20:32 pve kernel:  __handle_mm_fault+0xb93/0x1020
Oct 06 06:20:32 pve kernel:  ? prepend_copy+0x1a/0x60
Oct 06 06:20:32 pve kernel:  handle_mm_fault+0x136/0x3a0
Oct 06 06:20:32 pve kernel:  do_user_addr_fault+0x1c0/0x820
Oct 06 06:20:32 pve kernel:  exc_page_fault+0x81/0x1c0
Oct 06 06:20:32 pve kernel:  asm_exc_page_fault+0x27/0x30
Oct 06 06:20:32 pve kernel: RIP: 0010:_copy_to_iter+0x98/0x610
Oct 06 06:20:32 pve kernel: Code: 45 31 c9 48 01 d7 48 01 f9 41 0f 92 c1 48 be 00 f0 ff ff ff 7f 00 00 48 39 ce 72 75
Oct 06 06:20:32 pve kernel: RSP: 0018:ffffccfee3693ac0 EFLAGS: 00050246
Oct 06 06:20:32 pve kernel: RAX: 00000000003ca000 RBX: 0000000000001000 RCX: 0000000000000010
Oct 06 06:20:32 pve kernel: RDX: 0000000000336000 RSI: ffff8c584457dff0 RDI: 00007600c0fa3000
Oct 06 06:20:32 pve kernel: RBP: ffffccfee3693b48 R08: ffff8c584457d000 R09: 0000000000000000
Oct 06 06:20:32 pve kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffffccfee3693c18
Oct 06 06:20:32 pve kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000001000
Oct 06 06:20:32 pve kernel:  ? shmem_get_folio_gfp+0x169/0x5e0
Oct 06 06:20:32 pve kernel:  copy_page_to_iter+0x9f/0x190
Oct 06 06:20:32 pve kernel:  shmem_file_read_iter+0x185/0x430
Oct 06 06:20:32 pve kernel:  vfs_read+0x274/0x390
Oct 06 06:20:32 pve kernel:  ksys_read+0x6f/0xf0
Oct 06 06:20:32 pve kernel:  __x64_sys_read+0x19/0x30
Oct 06 06:20:32 pve kernel:  x64_sys_call+0x1ff1/0x2390
Oct 06 06:20:32 pve kernel:  do_syscall_64+0x10b/0x14e0
Oct 06 06:20:32 pve kernel:  ? putname+0x41/0x90
Oct 06 06:20:32 pve kernel:  ? do_sys_openat2+0x9e/0xf0
Oct 06 06:20:32 pve kernel:  ? __x64_sys_openat+0x52/0xa0
Oct 06 06:20:32 pve kernel:  ? x64_sys_call+0x1f4c/0x2390
Oct 06 06:20:32 pve kernel:  ? do_syscall_64+0x148/0x14e0
Oct 06 06:20:32 pve kernel:  ? do_user_addr_fault+0x1e7/0x820
Oct 06 06:20:32 pve kernel:  ? irqentry_exit+0xb2/0x710
Oct 06 06:20:32 pve kernel:  ? clear_bhb_loop+0x70/0xc0
Oct 06 06:20:32 pve kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Oct 06 06:20:32 pve kernel: RIP: 0033:0x7600c1e663a7
Oct 06 06:20:32 pve kernel: Code: Unable to access opcode bytes at 0x7600c1e6637d.
Oct 06 06:20:32 pve kernel: RSP: 002b:00007fff6a1ab900 EFLAGS: 00000202 ORIG_RAX: 0000000000000000
Oct 06 06:20:32 pve kernel: RAX: ffffffffffffffda RBX: 00007600c1d6e780 RCX: 00007600c1e663a7
Oct 06 06:20:32 pve kernel: RDX: 0000000000700000 RSI: 00007600c0c6c010 RDI: 0000000000000003
Oct 06 06:20:32 pve kernel: RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
Oct 06 06:20:32 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007600c0c6c010
Oct 06 06:20:32 pve kernel: R13: 00007fff6a1acdaa R14: 00007fff6a1ab980 R15: 0000000000000000
Oct 06 06:20:32 pve kernel:  </TASK>
Oct 06 06:20:32 pve kernel: Memory cgroup min protection 0kB -- low protection 0kB
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 5239 (systemd) total-vm:23508kB, anon-rss:358>
Oct 06 06:20:32 pve kernel: file invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Oct 06 06:20:32 pve kernel: CPU: 11 UID: 100000 PID: 2604997 Comm: file Tainted: P           O        7.0.14-20-pve #>
Oct 06 06:20:32 pve kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Oct 06 06:20:32 pve kernel: Hardware name: ASUSTeK COMPUTER INC. To be filled by O.E.M./P12R-M Series, BIOS 1901 01/1>
Oct 06 06:20:32 pve kernel: Call Trace:
Oct 06 06:20:32 pve kernel:  <TASK>
Oct 06 06:20:32 pve kernel:  dump_stack_lvl+0x5f/0x90
Oct 06 06:20:32 pve kernel:  dump_stack+0x10/0x18
Oct 06 06:20:32 pve kernel:  dump_header+0x48/0x1c7
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel:  out_of_memory+0x22f/0x4d0
Oct 06 06:20:32 pve kernel:  mem_cgroup_out_of_memory+0x100/0x120
Oct 06 06:20:32 pve kernel:  try_charge_memcg+0x3e6/0x650
Oct 06 06:20:32 pve kernel:  charge_memcg+0x34/0x90
Oct 06 06:20:32 pve kernel:  __mem_cgroup_charge+0x2d/0xa0
Oct 06 06:20:32 pve kernel:  do_anonymous_page+0x379/0x930
Oct 06 06:20:32 pve kernel:  ? __pte_offset_map+0x1c/0x130
Oct 06 06:20:32 pve kernel:  __handle_mm_fault+0xb93/0x1020
Oct 06 06:20:32 pve kernel:  ? prepend_copy+0x1a/0x60
Oct 06 06:20:32 pve kernel:  handle_mm_fault+0x136/0x3a0
Oct 06 06:20:32 pve kernel:  do_user_addr_fault+0x1c0/0x820
Oct 06 06:20:32 pve kernel:  exc_page_fault+0x81/0x1c0
Oct 06 06:20:32 pve kernel:  asm_exc_page_fault+0x27/0x30
Oct 06 06:20:32 pve kernel: RIP: 0010:_copy_to_iter+0x98/0x610
Oct 06 06:20:32 pve kernel: Code: 45 31 c9 48 01 d7 48 01 f9 41 0f 92 c1 48 be 00 f0 ff ff ff 7f 00 00 48 39 ce 72 75>
Oct 06 06:20:32 pve kernel: RSP: 0018:ffffccfee3693ac0 EFLAGS: 00050246
Oct 06 06:20:32 pve kernel: RAX: 00000000003ca000 RBX: 0000000000001000 RCX: 0000000000000010
Oct 06 06:20:32 pve kernel: RDX: 0000000000336000 RSI: ffff8c584457dff0 RDI: 00007600c0fa3000
Oct 06 06:20:32 pve kernel: RBP: ffffccfee3693b48 R08: ffff8c584457d000 R09: 0000000000000000
Oct 06 06:20:32 pve kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffffccfee3693c18
Oct 06 06:20:32 pve kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000001000
Oct 06 06:20:32 pve kernel:  ? shmem_get_folio_gfp+0x169/0x5e0
Oct 06 06:20:32 pve kernel:  copy_page_to_iter+0x9f/0x190
Oct 06 06:20:32 pve kernel:  shmem_file_read_iter+0x185/0x430
Oct 06 06:20:32 pve kernel:  vfs_read+0x274/0x390
Oct 06 06:20:32 pve kernel:  ksys_read+0x6f/0xf0
Oct 06 06:20:32 pve kernel:  __x64_sys_read+0x19/0x30
Oct 06 06:20:32 pve kernel:  x64_sys_call+0x1ff1/0x2390
Oct 06 06:20:32 pve kernel:  do_syscall_64+0x10b/0x14e0
Oct 06 06:20:32 pve kernel:  ? putname+0x41/0x90
Oct 06 06:20:32 pve kernel:  ? do_sys_openat2+0x9e/0xf0
Oct 06 06:20:32 pve kernel:  ? __x64_sys_openat+0x52/0xa0
Oct 06 06:20:32 pve kernel:  ? x64_sys_call+0x1f4c/0x2390
Oct 06 06:20:32 pve kernel:  ? do_syscall_64+0x148/0x14e0
Oct 06 06:20:32 pve kernel:  ? do_user_addr_fault+0x1e7/0x820
Oct 06 06:20:32 pve kernel:  ? irqentry_exit+0xb2/0x710
Oct 06 06:20:32 pve kernel:  ? clear_bhb_loop+0x70/0xc0
Oct 06 06:20:32 pve kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Oct 06 06:20:32 pve kernel: RIP: 0033:0x7600c1e663a7
Oct 06 06:20:32 pve kernel: Code: Unable to access opcode bytes at 0x7600c1e6637d.
Oct 06 06:20:32 pve kernel: RSP: 002b:00007fff6a1ab900 EFLAGS: 00000202 ORIG_RAX: 0000000000000000
Oct 06 06:20:32 pve kernel: RAX: ffffffffffffffda RBX: 00007600c1d6e780 RCX: 00007600c1e663a7
Oct 06 06:20:32 pve kernel: RDX: 0000000000700000 RSI: 00007600c0c6c010 RDI: 0000000000000003
Oct 06 06:20:32 pve kernel: RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
Oct 06 06:20:32 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007600c0c6c010
Oct 06 06:20:32 pve kernel: R13: 00007fff6a1acdaa R14: 00007fff6a1ab980 R15: 0000000000000000
Oct 06 06:20:32 pve kernel:  </TASK>
Oct 06 06:20:32 pve kernel: Memory cgroup min protection 0kB -- low protection 0kB
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 2604996 (bash) total-vm:7352kB, anon-rss:1036>
Oct 06 06:20:32 pve kernel: dbus-daemon invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_sco>
Oct 06 06:20:32 pve kernel: CPU: 11 UID: 100102 PID: 5584 Comm: dbus-daemon Tainted: P           O        7.0.14-20-p>
Oct 06 06:20:32 pve kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Oct 06 06:20:32 pve kernel: Hardware name: ASUSTeK COMPUTER INC. To be filled by O.E.M./P12R-M Series, BIOS 1901 01/1>
Oct 06 06:20:32 pve kernel: Call Trace:
...

How can I understand this message ?

IA asked me the following command for investigation :

Code: 
dmesg -T | grep -i -E "segfault|page fault|opcode|instruction"
[Sun Oct  4 14:36:23 2026] x86/cpu: User Mode Instruction Prevention (UMIP) activated
[Tue Oct  6 06:20:32 2026] Code: Unable to access opcode bytes at 0x7600c1e6637d.
[Tue Oct  6 06:20:32 2026] Code: Unable to access opcode bytes at 0x7600c1e6637d.
[Tue Oct  6 06:20:32 2026] Code: Unable to access opcode bytes at 0x76e6d673937d.
[Tue Oct  6 06:20:32 2026] Code: Unable to access opcode bytes at 0x7600c1e6637d.

IA proposed this modification in /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_idle.max_cstate=1 processor.max_cstate=1"

I know that I update pve via the menu update last weekend.

Today I didn't modify grub file. I would like to know risk and what is this error.

My installation :
Code: 
proxmox-ve: 9.2.0 (running kernel: 7.0.14-20-pve)
pve-manager: 9.2.21 (running version: 9.2.21/4f6e0ac86f9e8c7f)
proxmox-kernel-helper: 9.2.0
proxmox-kernel-7.0: 7.0.14-20
proxmox-kernel-7.0.14-20-pve-signed: 7.0.14-20
proxmox-kernel-7.0.14-19-pve-signed: 7.0.14-19
proxmox-kernel-7.0.14-17-pve-signed: 7.0.14-17
proxmox-kernel-7.0.14-16-pve-signed: 7.0.14-16
proxmox-kernel-7.0.14-15-pve-signed: 7.0.14-15
proxmox-kernel-7.0.14-14-pve-signed: 7.0.14-14
proxmox-kernel-7.0.14-12-pve-signed: 7.0.14-12
proxmox-kernel-7.0.14-6-pve-signed: 7.0.14-6
proxmox-kernel-7.0.12-1-pve-signed: 7.0.12-1
proxmox-kernel-7.0.6-2-pve-signed: 7.0.6-2
proxmox-kernel-7.0.2-6-pve-signed: 7.0.2-6
proxmox-kernel-7.0.2-4-pve-signed: 7.0.2-4
proxmox-kernel-7.0.2-2-pve-signed: 7.0.2-2
proxmox-kernel-7.0.0-3-pve-signed: 7.0.0-3
proxmox-kernel-6.17: 6.17.13-21
proxmox-kernel-6.17.13-21-pve-signed: 6.17.13-21
proxmox-kernel-6.17.13-19-pve-signed: 6.17.13-19
proxmox-kernel-6.17.13-13-pve-signed: 6.17.13-13
proxmox-kernel-6.17.13-11-pve-signed: 6.17.13-11
proxmox-kernel-6.17.13-9-pve-signed: 6.17.13-9
proxmox-kernel-6.17.13-7-pve-signed: 6.17.13-7
proxmox-kernel-6.17.13-6-pve-signed: 6.17.13-6
proxmox-kernel-6.17.13-4-pve-signed: 6.17.13-4
proxmox-kernel-6.17.13-2-pve-signed: 6.17.13-2
proxmox-kernel-6.17.13-1-pve-signed: 6.17.13-1
proxmox-kernel-6.17.9-1-pve-signed: 6.17.9-1
proxmox-kernel-6.17.4-2-pve-signed: 6.17.4-2
proxmox-kernel-6.17.4-1-pve-signed: 6.17.4-1
proxmox-kernel-6.17.2-2-pve-signed: 6.17.2-2
proxmox-kernel-6.17.2-1-pve-signed: 6.17.2-1
proxmox-kernel-6.14: 6.14.11-9
proxmox-kernel-6.14.11-9-pve-signed: 6.14.11-9
proxmox-kernel-6.14.11-8-pve-signed: 6.14.11-8
proxmox-kernel-6.14.11-7-pve-signed: 6.14.11-7
proxmox-kernel-6.14.11-6-pve-signed: 6.14.11-6
proxmox-kernel-6.14.11-5-pve-signed: 6.14.11-5
proxmox-kernel-6.14.11-4-pve-signed: 6.14.11-4
proxmox-kernel-6.14.11-3-pve-signed: 6.14.11-3
proxmox-kernel-6.14.8-2-pve-signed: 6.14.8-2
ceph-fuse: 19.2.6-pve4
corosync: 3.1.10-pve3
criu: 4.1.1-1
frr-pythontools: 10.6.1-1+pve3
ifupdown2: 3.3.0-1+pmx12
intel-microcode: 3.20251111.1~deb13u1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-7
libproxmox-acme-perl: 1.7.2
libproxmox-backup-qemu0: 2.0.3
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.1.2
libpve-apiclient-perl: 3.4.3
libpve-cluster-api-perl: 9.1.6
libpve-cluster-perl: 9.1.6
libpve-common-perl: 9.2.2
libpve-guest-common-perl: 6.0.5
libpve-http-server-perl: 6.0.5
libpve-network-perl: 1.6.7
libpve-notify-perl: 9.1.6
libpve-rs-perl: 0.15.3
libpve-storage-perl: 9.1.11
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 7.0.0-2
lxcfs: 7.0.0-pve1
novnc-pve: 1.7.0-2
proxmox-backup-client: 4.2.7-1
proxmox-backup-file-restore: 4.2.7-1
proxmox-backup-restore-image: 1.0.0
proxmox-enterprise-support-keyring: 1.1
proxmox-firewall: 1.2.3
proxmox-kernel-helper: 9.2.0
proxmox-mail-forward: 1.0.3
proxmox-mini-journalreader: 1.7
proxmox-offline-mirror-helper: 0.7.4
proxmox-widget-toolkit: 5.2.10
pve-cluster: 9.1.6
pve-container: 6.1.14
pve-docs: 9.2.13
pve-edk2-firmware: 4.2026.08-1
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.6
pve-firmware: 3.18-6
pve-ha-manager: 5.2.5
pve-i18n: 3.10.0
pve-qemu-kvm: 11.0.3-4
pve-xtermjs: 6.0.0-2
qemu-server: 9.2.10
smartmontools: 7.5-pve2
spiceterm: 3.4.2
swtpm: 0.8.0+pve3
vncterm: 1.9.2
zfsutils-linux: 2.4.4-pve1


Thanks for your advice
 
Code: 
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 2604996 (bash) total-vm:7352kB, anon-rss:1036>
Oct 06 06:20:32 pve kernel: dbus-daemon invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_sco>
These lines mean your system ran out of memory - the oom-killer daemon killed a random process on your system to prevent a full system crash / hang.

Check if you have not overcommitted memory on your system. PVE needs at least 2GB+ for itself. PVE, VM's and containers memory usage combined have to be lower than the installed ram on your system, keep some GB free just to be on the save side. Depending on the used file system (ZFS?) and other factors (caching ...) you may need even more spare RAM.

You have many old kernels installed, you could remove them with apt-get autoremove
You may have to remove the meta packages proxmox-kernel-6.14 and proxmox-kernel-6.17
and do another apt-get autoremove to get rid of the older kernels if you don't need them anymore.
 
Some of your lines are cut off. Please use a proper SSH client to copy from. UID: 100000 leads me to believe this happened inside a CT. If you don't cut off the lines we should be able to tell which one it was. I would not listen to "IA" and hold off on modifying the kernel args. I see no relation here.
Also share this
Bash: 
journalctl -rg "oom|killed"
 
Last edited:
Hi,

Here the results of the command journalctl -rg "oom|killed"

Code: 
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 2604725 (bash) total-vm:7352kB, anon-rss:344kB, file-rss:8kB, shmem-rss:0kB, UID:100000 pgtables:56kB oom_score_adj:0
Oct 06 06:20:32 pve kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=ns,mems_allowed=0,oom_memcg=/lxc/118,task_memcg=/lxc/118/ns/system.slice/pulse-update.service,task=bash,pid=2604725,uid=100000
Oct 06 06:20:32 pve kernel: [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel: file invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 2604995 (bash) total-vm:7352kB, anon-rss:444kB, file-rss:0kB, shmem-rss:0kB, UID:100000 pgtables:56kB oom_score_adj:0
Oct 06 06:20:32 pve kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=ns,mems_allowed=0,oom_memcg=/lxc/118,task_memcg=/lxc/118/ns/system.slice/pulse-update.service,task=bash,pid=2604995,uid=100000
Oct 06 06:20:32 pve kernel: [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel: dbus-daemon invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 2604996 (bash) total-vm:7352kB, anon-rss:1036kB, file-rss:0kB, shmem-rss:0kB, UID:100000 pgtables:56kB oom_score_adj:0
Oct 06 06:20:32 pve kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=ns,mems_allowed=0,oom_memcg=/lxc/118,task_memcg=/lxc/118/ns/system.slice/pulse-update.service,task=bash,pid=2604996,uid=100000
Oct 06 06:20:32 pve kernel: [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel: file invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Oct 06 06:20:32 pve kernel: Memory cgroup out of memory: Killed process 5239 (systemd) total-vm:23508kB, anon-rss:3580kB, file-rss:8kB, shmem-rss:0kB, UID:100000 pgtables:88kB oom_score_adj:0
Oct 06 06:20:32 pve kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=ns,mems_allowed=0,oom_memcg=/lxc/118,task_memcg=/lxc/118/ns/init.scope,task=systemd,pid=5239,uid=100000
Oct 06 06:20:32 pve kernel: [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
Oct 06 06:20:32 pve kernel:  oom_kill_process.cold+0x8/0x87
Oct 06 06:20:32 pve kernel: file invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
-- Boot 50251c5a33574c0795f197b4a4ab62cf --

I will look for autoremove.

My LXC and VM have a cumulated total of RAM at 120GB, for a total RAM of 128GB. Perhaps too much. I will see if I can decrease some VM or LXC.

My Proxmox is installed on RAIDZ mirror (ZFS).

Thanks
 
There we go. It's caused by things inside CT 118. Can you run something like this inside that CT?
Bash: 
top -cem -o%MEM
For CTs the memory is just a quota so over-allocating isn't as bad as it is with VMs.
Seeing the config of the CT would be of interest as well. Based on the current data my assumption is that you limited the memory of 118 too much.
Bash: 
pct config 118
I'd also be interested in seeing the node's Summary.
 
Last edited:
Thanks Impact.

Code: 
top - 21:17:42 up 5 days,  6:41,  1 user,  load average: 1.69, 2.07, 1.98
Tasks: 670 total,   1 running, 669 sleeping,   0 stopped,   0 zombie
%Cpu(s):  8.4 us,  9.9 sy,  0.0 ni, 81.7 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem : 128721.4 total,  11448.4 free,  94582.3 used,  25034.6 buff/cache 
MiB Swap: 114473.5 total, 114260.2 free,    213.2 used.  34139.2 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                   
   3219 root      20   0   65.9g  64.1g   7.8m S   1.3  51.0     13,24 /usr/bin/kvm -id 121 -name Xigmanas1 -no-shut+
   6333 root      20   0 9115.5m   8.1g   8.0m S  30.2   6.4     39,29 /usr/bin/kvm -id 115 -name Lubuntu24 -no-shut+
   4966 root      20   0   13.0g   6.2g   6.3m S  51.2   4.9     37,45 /usr/bin/kvm -id 109 -name HAOS -no-shutdown +
   5077 root      20   0 4822.6m   1.1g   6.3m S   1.0   0.9  51:23.77 /usr/bin/kvm -id 111 -name Scrutiny -no-shutd+
2667923 100999    20   0   27.4g 575.3m  58.9m S   0.0   0.4  15:19.87 immich                                     
2668172 100999    20   0    9.8g 280.2m  45.9m S   0.0   0.2   0:17.49 immich-api                                 
   3231 100000    20   0 1739.3m 244.5m  17.4m S   0.0   0.2  31:23.25 /opt/AdGuardHome/AdGuardHome -s run       
   2941 100000    20   0  387.6m 228.2m 226.9m S   0.0   0.2   0:44.55 /usr/lib/systemd/systemd-journald         
4116589 root      20   0  282.7m 206.9m   9.9m S   3.7   0.2   0:14.13 pvedaemon worker                           
4122898 root      20   0  272.7m 197.2m  10.9m S   1.0   0.2   0:09.56 pvedaemon worker                           
4127986 root      20   0  272.5m 195.5m   9.3m S   0.0   0.2   0:06.34 pvedaemon worker                           
2662297 100999    20   0 1554.8m 195.1m  29.7m S   0.3   0.2  28:10.92 /opt/pulse/bin/pulse                       
   2770 www-data  20   0  227.5m 169.4m  20.3m S   0.0   0.1   0:15.16 pveproxy                                   
   2099 root      20   0  616.0m 161.0m  10.1m S   0.3   0.1  32:58.11 /opt/glances/.venv/bin/python3 /opt/glances/.+
4127109 www-data  20   0  237.4m 160.7m   8.7m S   0.3   0.1   0:02.59 pveproxy worker                           
  38104 www-data  20   0  237.3m 159.6m   7.9m S   1.3   0.1   0:00.51 pveproxy worker                           
  23635 www-data  20   0  237.2m 159.5m   7.9m S   1.0   0.1   0:00.68 pveproxy worker                           
4104945 root      20   0  239.7m 157.8m   3.3m S   0.0   0.1   0:00.05 task UPID:pve:003EA2F1:02B55B1F:6AC93744:vncs+
4115789 www-data  20   0  237.5m 155.8m   3.7m S   0.3   0.1   0:00.66 pveproxy worker (shutdown)                 
   2758 root      20   0  226.2m 150.8m   2.8m S   0.0   0.1   0:07.30 pvedaemon

It is my pulse LXC (LXC 118)

Code: 
arch: amd64
cores: 2
cpulimit: 2
features: nesting=1
hostname: pulse
memory: 1024
net0: name=eth0,bridge=vmbr1,firewall=0,gw=192.168.XXXXX,hwaddr=XXXX,ip=192.168.XXXX,type=veth
onboot: 1
ostype: debian
protection: 1
rootfs: vmdata:subvol-118-disk-0,size=8G
startup: order=5
swap: 256
unprivileged: 1

My swap for PVE is on SSD disk.

And for the summary you can see in the attach file (sorry in French language)
 

Attachments

  • Capture d’écran_2026-10-09_21-23-13.png
    Capture d’écran_2026-10-09_21-23-13.png
    63.2 KB · Views: 3
Last edited:
The node seems healthy, memory wise. I wanted to see top -cem -o%MEM from inside the 118 CT. Can you share that too?
 
Inside LXC 118 (Pulse)


Code: 
top - 19:45:01 up 2 min,  0 users,  load average: 1.05, 1.08, 0.51
Tasks:  16 total,   1 running,  15 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.1 us,  2.9 sy,  0.0 ni, 94.6 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
MiB Mem :   1024.0 total,    862.2 free,    106.5 used,     55.3 buff/cache     
MiB Swap:    256.0 total,    256.0 free,      0.0 used.    917.5 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                       
    140 pulse     20   0 1417.5m 109.9m  25.5m S   1.3  10.7   0:01.93 /opt/pulse/bin/pulse                           
      1 root      20   0   22.5m   9.6m   6.5m S   0.0   0.9   0:00.12 /sbin/init                                     
     49 root      20   0   41.8m   8.6m   7.4m S   0.0   0.8   0:00.06 /usr/lib/systemd/systemd-journald             
     89 systemd+  20   0   20.1m   7.3m   6.0m S   0.0   0.7   0:00.02 /usr/lib/systemd/systemd-networkd             
    142 root      20   0   18.0m   6.3m   5.2m S   0.0   0.6   0:00.03 /usr/lib/systemd/systemd-logind               
    153 root      20   0   11.5m   4.8m   3.7m S   0.0   0.5   0:00.00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-10+
    285 postfix   20   0   43.4m   4.5m   3.8m S   0.0   0.4   0:00.00 pickup -l -t unix -u -c                       
    286 postfix   20   0   43.5m   4.4m   3.8m S   0.0   0.4   0:00.00 qmgr -l -t unix -u                             
    345 root      20   0    5.6m   4.1m   2.2m S   0.0   0.4   0:00.00 -bash                                         
    358 root      20   0    7.2m   3.9m   1.9m R   0.0   0.4   0:00.00 top -cem -o%MEM                               
    149 root      20   0    7.5m   3.1m   2.5m S   0.0   0.3   0:00.00 login -- root                                 
    136 message+  20   0    8.2m   3.0m   2.5m S   0.0   0.3   0:00.00 /usr/bin/dbus-daemon --system --address=syste+
    284 root      20   0   42.9m   2.8m   2.1m S   0.0   0.3   0:00.00 /usr/lib/postfix/sbin/master -w               
    150 root      20   0    4.7m   1.6m   1.4m S   0.0   0.2   0:00.00 /sbin/agetty -o -- \u --noreset --noclear - l+
    148 root      20   0    4.7m   1.6m   1.4m S   0.0   0.2   0:00.00 /sbin/agetty -o -- \u --noreset --noclear --k+
    135 root      20   0    3.8m   1.5m   1.2m S   0.0   0.1   0:00.00 /usr/sbin/cron -f

Thanks for your support
 
Hmm. That looks normal at the moment. I can only assume that pulse-update.service does something suspect. I'd check this
Bash: 
systemctl status pulse-update.service
systemctl cat pulse-update.service
journalctl -u pulse-update.service
Maybe you can even reproduce it by triggering it manually and monitor while having top open but let's look at what it does first.
Is anything else, maybe timer/cron based, running in there besides pulse?
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back