Low Memory Corruption

[OXIB]

Hello,
Running Proxmox on this box with 32Gigs of RAM and ZFS.
Today I logged in and all the VMs and the server had a question mark on it.
I rebooted the server and it took a while to restart. I looked at the syslog and found the low memory corruption entry. Can anyone please take a look at the dmesg log and let me know if something critical is going on?
The BIOS on the motherboard is running the latest and there has not been an update since 2018.
Thanks very much for any help you can provide.:

 

[Moayad]

Hello,

Please post the full output of pveversion -v and what say the output of systemctl status pve-cluster.service and systemctl status pveproxy.service

 

[OXIB]

Hello,

Please post the full output of pveversion -v and what say the output of systemctl status pve-cluster.service and systemctl status pveproxy.service

Thanks very much for you reply Moayad, below is the full output of pveversion -v:
proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve)
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)
pve-kernel-5.4: 6.3-3
pve-kernel-helper: 6.3-3
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.73-1-pve: 5.4.73-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-2-pve: 4.13.13-33
ceph-fuse: 12.2.13-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-2
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-3
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.6-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-3
pve-cluster: 6.2-1
pve-container: 3.3-2
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-7
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1

Here is systemctl status pve-cluster.service:
● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2020-12-29 14:53:20 EST; 20h ago
Process: 2302 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
Main PID: 2303 (pmxcfs)
Tasks: 7 (limit: 4915)
Memory: 55.7M
CGroup: /system.slice/pve-cluster.service
└─2303 /usr/bin/pmxcfs

Dec 29 14:53:19 VMSERVER systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 29 14:53:20 VMSERVER systemd[1]: Started The Proxmox VE cluster filesystem.

And here is systemctl status pveproxy.service:
● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2020-12-29 14:53:27 EST; 20h ago
Process: 2463 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 2465 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Process: 30704 ExecReload=/usr/bin/pveproxy restart (code=exited, status=0/SUCCESS)
Main PID: 2568 (pveproxy)
Tasks: 5 (limit: 4915)
Memory: 171.4M
CGroup: /system.slice/pveproxy.service
├─ 2568 pveproxy
├─ 3246 pveproxy worker
├─11388 pveproxy worker
├─12548 pveproxy worker (shutdown)
└─26210 pveproxy worker

Dec 30 10:38:07 VMSERVER pveproxy[2568]: starting 1 worker(s)
Dec 30 10:38:07 VMSERVER pveproxy[2568]: worker 26210 started
Dec 30 10:52:20 VMSERVER pveproxy[30735]: worker exit
Dec 30 10:52:20 VMSERVER pveproxy[2568]: worker 30735 finished
Dec 30 10:52:20 VMSERVER pveproxy[2568]: starting 1 worker(s)
Dec 30 10:52:20 VMSERVER pveproxy[2568]: worker 11388 started
Dec 30 10:58:00 VMSERVER pveproxy[12549]: worker exit
Dec 30 10:58:00 VMSERVER pveproxy[2568]: worker 12549 finished
Dec 30 10:58:00 VMSERVER pveproxy[2568]: starting 1 worker(s)
Dec 30 10:58:00 VMSERVER pveproxy[2568]: worker 3246 started

 

[Moayad]

Please try to generate a new PVE certificate by doing the following command on your node: pvecm updatecerts -f then try to re-login.

If that not help please post the output of journalctl -u pveproxy

 

[OXIB]

Please try to generate a new PVE certificate by doing the following command on your node: pvecm updatecerts -f then try to re-login.

If that not help please post the output of journalctl -u pveproxy

Apologies for replying until now but with the year end holidays it was busy.
I entered the pvecm updatecerts -f command and it updated them.
Rebooted and the low memory entry is still showing in the syslog.
So I ran the journalctl -u pveproxy and this is what I got:
-- Logs begin at Sun 2021-01-03 16:32:30 EST, end at Sun 2021-01-03 16:53:05 EST. --
Jan 03 16:32:48 VMSERVER systemd[1]: Starting PVE API Proxy Server...
Jan 03 16:32:49 VMSERVER pveproxy[2603]: starting server
Jan 03 16:32:49 VMSERVER pveproxy[2603]: starting 3 worker(s)
Jan 03 16:32:49 VMSERVER pveproxy[2603]: worker 2604 started
Jan 03 16:32:49 VMSERVER pveproxy[2603]: worker 2605 started
Jan 03 16:32:49 VMSERVER pveproxy[2603]: worker 2606 started
Jan 03 16:32:50 VMSERVER systemd[1]: Started PVE API Proxy Server.


Thanks very much.

 

[Moayad]

Hello again,


Could you post the full output of journalctl -b as attach.

 

[mmenaz]

Hello,
Running Proxmox on this box with 32Gigs of RAM and ZFS.
Today I logged in and all the VMs and the server had a question mark on it.
I rebooted the server and it took a while to restart. I looked at the syslog and found the low memory corruption entry. Can anyone please take a look at the dmesg log and let me know if something critical is going on?
The BIOS on the motherboard is running the latest and there has not been an update since 2018.
Thanks very much for any help you can provide.:

if you mean

[    0.013208] check: Scanning 1 areas for low memory corruption

I really thing is not an error, just a notification that is scanning (checking) IF there is low memory corruption.

 

[OXIB]

Hello Moayad - here is the output of journalctl -b.
Thanks very much.

 

[OXIB]

if you mean

[    0.013208] check: Scanning 1 areas for low memory corruption

I really thing is not an error, just a notification that is scanning (checking) IF there is low memory corruption.

Hi - I hope it is not an error but it's always good to check just to make sure.