Hello !
I encounter slow system and out of memory situations on my server running latest Proxmox version. I do use ZFS with 16 Go ram, first installed with PVE5 then upgraded, with a few running CT's only. I tried to cap the memory used with "options zfs zfs_arc_max" with no luck
proxmox-ve: 6.2-2 (running kernel: 5.4.73-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
It seems the pveproxy processes use 1 GB more every few hours... There is a lot of stuff in the log (see below), but searching in the forum was not helpful.
Can you help ?
# arcstat
time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c
11:13:30 13 0 0 0 0 0 0 0 0 3.1G 8.0G
# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 1,81T 1,26T 566G - - 37% 69% 1.03x ONLINE -
# service pveproxy status
● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2020-11-24 19:13:57 CET; 15h ago
Process: 30167 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 30176 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Process: 8408 ExecReload=/usr/bin/pveproxy restart (code=exited, status=0/SUCCESS)
Main PID: 30184 (pveproxy)
Tasks: 600 (limit: 4915)
Memory: 3.9G
CGroup: /system.slice/pveproxy.service
├─ 327 pveproxy worker (shutdown)
(...)
├─ 2415 pveproxy worker (shutdown)
├─ 2416 pveproxy worker
├─ 2450 pveproxy worker (shutdown)
(...)
lines 1-80
# journalctl -u pveproxy
nov. 24 09:55:41 x pveproxy[2731]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.
nov. 24 09:55:41 x pveproxy[2729]: worker 2731 finished
nov. 24 09:55:41 x pveproxy[2729]: starting 1 worker(s)
nov. 24 09:55:41 x pveproxy[2729]: worker 6367 started
nov. 24 09:55:44 x pveproxy[6356]: got inotify poll request in wrong process - disabling inotify
nov. 24 09:56:43 x pveproxy[2730]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.
nov. 24 09:56:43 x pveproxy[2729]: worker 2730 finished
nov. 24 09:56:43 x pveproxy[2729]: starting 1 worker(s)
nov. 24 09:56:43 x pveproxy[2729]: worker 19065 started
nov. 24 09:56:44 x pveproxy[19054]: got inotify poll request in wrong process - disabling inotify
nov. 24 09:58:44 x pveproxy[19065]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.
I encounter slow system and out of memory situations on my server running latest Proxmox version. I do use ZFS with 16 Go ram, first installed with PVE5 then upgraded, with a few running CT's only. I tried to cap the memory used with "options zfs zfs_arc_max" with no luck
proxmox-ve: 6.2-2 (running kernel: 5.4.73-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
It seems the pveproxy processes use 1 GB more every few hours... There is a lot of stuff in the log (see below), but searching in the forum was not helpful.
Can you help ?
# arcstat
time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c
11:13:30 13 0 0 0 0 0 0 0 0 3.1G 8.0G
# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 1,81T 1,26T 566G - - 37% 69% 1.03x ONLINE -
# service pveproxy status
● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2020-11-24 19:13:57 CET; 15h ago
Process: 30167 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 30176 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Process: 8408 ExecReload=/usr/bin/pveproxy restart (code=exited, status=0/SUCCESS)
Main PID: 30184 (pveproxy)
Tasks: 600 (limit: 4915)
Memory: 3.9G
CGroup: /system.slice/pveproxy.service
├─ 327 pveproxy worker (shutdown)
(...)
├─ 2415 pveproxy worker (shutdown)
├─ 2416 pveproxy worker
├─ 2450 pveproxy worker (shutdown)
(...)
lines 1-80
# journalctl -u pveproxy
nov. 24 09:55:41 x pveproxy[2731]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.
nov. 24 09:55:41 x pveproxy[2729]: worker 2731 finished
nov. 24 09:55:41 x pveproxy[2729]: starting 1 worker(s)
nov. 24 09:55:41 x pveproxy[2729]: worker 6367 started
nov. 24 09:55:44 x pveproxy[6356]: got inotify poll request in wrong process - disabling inotify
nov. 24 09:56:43 x pveproxy[2730]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.
nov. 24 09:56:43 x pveproxy[2729]: worker 2730 finished
nov. 24 09:56:43 x pveproxy[2729]: starting 1 worker(s)
nov. 24 09:56:43 x pveproxy[2729]: worker 19065 started
nov. 24 09:56:44 x pveproxy[19054]: got inotify poll request in wrong process - disabling inotify
nov. 24 09:58:44 x pveproxy[19065]: Can't call method "binip" on an undefined value at /usr/share/perl5/Net/IP.pm line 757.