Used memory much greater than what is used by processes

Zeash

Member
Jan 23, 2022
11
0
6
20
Hello,

Ever since I've cloned my old PBS's SSD to a new one and moved it to newer hardware, I've noticed that "used" RAM (putting aside ARC) is significantly greater than what it should be when adding up that which the processes use.


1725577191655.png

Bash:
root@pbs-0:~# ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -n
0 MB            [arc_evict]
0 MB            [arc_prune]
0 MB            [arc_reap]
0 MB            [card0-crtc0]
0 MB            [card0-crtc1]
0 MB            [card0-crtc2]
0 MB            COMMAND
0 MB            [cpuhp/0]
0 MB            [cpuhp/1]
0 MB            [cpuhp/2]
0 MB            [cpuhp/3]
0 MB            [dbu_evict]
0 MB            [dbuf_evict]
0 MB            [dp_sync_taskq]
0 MB            [dp_sync_taskq]
0 MB            [dp_sync_taskq]
0 MB            [dp_zil_clean_ta]
0 MB            [dp_zil_clean_ta]
0 MB            [dp_zil_clean_ta]
0 MB            [dp_zil_clean_ta]
0 MB            [ecryptfs-kthread]
0 MB            [hwrng]
0 MB            [idle_inject/0]
0 MB            [idle_inject/1]
0 MB            [idle_inject/2]
0 MB            [idle_inject/3]
0 MB            [irq/120-aerdrv]
0 MB            [irq/121-aerdrv]
0 MB            [irq/130-mei_me]
0 MB            [irq/9-acpi]
0 MB            [jbd2/dm-1-8]
0 MB            [kauditd]
0 MB            [kcompactd0]
0 MB            [kdevtmpfs]
0 MB            [khugepaged]
0 MB            [khungtaskd]
0 MB            [ksmd]
0 MB            [ksoftirqd/0]
0 MB            [ksoftirqd/1]
0 MB            [ksoftirqd/2]
0 MB            [ksoftirqd/3]
0 MB            [kswapd0]
0 MB            [kthreadd]
0 MB            [kworker/0:0-cgroup_destroy]
0 MB            [kworker/0:0H-events_highpri]
0 MB            [kworker/0:1-events]
0 MB            [kworker/0:1H-kblockd]
0 MB            [kworker/0:2-rcu_gp]
0 MB            [kworker/1:0H-events_highpri]
0 MB            [kworker/1:1-events]
0 MB            [kworker/1:1H-kblockd]
0 MB            [kworker/1:2-i915-unordered]
0 MB            [kworker/2:0H-events_highpri]
0 MB            [kworker/2:0-i915-unordered]
0 MB            [kworker/2:1H-kblockd]
0 MB            [kworker/2:1-i915-unordered]
0 MB            [kworker/2:2-events]
0 MB            [kworker/3:0-events]
0 MB            [kworker/3:0H-kblockd]
0 MB            [kworker/3:1-events]
0 MB            [kworker/3:1H]
0 MB            [kworker/3:2-events]
0 MB            [kworker/R-acpi_]
0 MB            [kworker/R-ata_s]
0 MB            [kworker/R-blkcg]
0 MB            [kworker/R-charg]
0 MB            [kworker/R-crypt]
0 MB            [kworker/R-devfr]
0 MB            [kworker/R-edac-]
0 MB            [kworker/R-ext4-]
0 MB            [kworker/R-inet_]
0 MB            [kworker/R-ipv6_]
0 MB            [kworker/R-kbloc]
0 MB            [kworker/R-kdmfl]
0 MB            [kworker/R-kdmfl]
0 MB            [kworker/R-kinte]
0 MB            [kworker/R-kmems]
0 MB            [kworker/R-kstrp]
0 MB            [kworker/R-kthro]
0 MB            [kworker/R-md]
0 MB            [kworker/R-md_bi]
0 MB            [kworker/R-mld]
0 MB            [kworker/R-mm_pe]
0 MB            [kworker/R-netns]
0 MB            [kworker/R-nvme-]
0 MB            [kworker/R-nvme-]
0 MB            [kworker/R-nvme-]
0 MB            [kworker/R-nvme-]
0 MB            [kworker/R-rcu_g]
0 MB            [kworker/R-rcu_p]
0 MB            [kworker/R-rpcio]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-scsi_]
0 MB            [kworker/R-slub_]
0 MB            [kworker/R-tls-s]
0 MB            [kworker/R-tpm_d]
0 MB            [kworker/R-ttm]
0 MB            [kworker/R-write]
0 MB            [kworker/R-xprti]
0 MB            [kworker/u8:0-flush-252:1]
0 MB            [kworker/u8:1-flush-252:1]
0 MB            [kworker/u8:2-events_unbound]
0 MB            [kworker/u8:3-events_power_efficient]
0 MB            [kworker/u8:4-ext4-rsv-conversion]
0 MB            [kworker/u9:0]
0 MB            [l2arc_feed]
0 MB            [migration/0]
0 MB            [migration/1]
0 MB            [migration/2]
0 MB            [migration/3]
0 MB            [mmp]
0 MB            [oom_reaper]
0 MB            [pool_workqueue_release]
0 MB            [rcu_preempt]
0 MB            [rcu_tasks_kthread]
0 MB            [rcu_tasks_rude_kthread]
0 MB            [rcu_tasks_trace_kthread]
0 MB            [scsi_eh_0]
0 MB            [scsi_eh_1]
0 MB            [scsi_eh_2]
0 MB            [scsi_eh_3]
0 MB            [scsi_eh_4]
0 MB            [scsi_eh_5]
0 MB            [spl_delay_taskq]
0 MB            [spl_dynamic_tas]
0 MB            [spl_kmem_cache]
0 MB            [spl_system_task]
0 MB            [txg_quiesce]
0 MB            [txg_sync]
0 MB            [watchdogd]
0 MB            [z_checkpoint_di]
0 MB            [z_cl_int]
0 MB            [z_cl_iss]
0 MB            [z_fr_int]
0 MB            [z_fr_iss]
0 MB            [z_indirect_cond]
0 MB            [z_ioctl_int]
0 MB            [z_ioctl_iss]
0 MB            [z_livelist_cond]
0 MB            [z_livelist_dest]
0 MB            [z_metaslab]
0 MB            [z_metaslab]
0 MB            [z_null_int]
0 MB            [z_null_iss]
0 MB            [z_prefetch]
0 MB            [z_rd_int]
0 MB            [z_rd_int]
0 MB            [z_rd_iss]
0 MB            [z_trim_int]
0 MB            [z_trim_iss]
0 MB            [z_unlinked_drai]
0 MB            [z_upgrade]
0 MB            [z_vdev_file]
0 MB            [zvol_tq-0]
0 MB            [z_wr_int]
0 MB            [z_wr_int]
0 MB            [z_wr_int]
0 MB            [z_wr_int_h]
0 MB            [z_wr_int_h]
0 MB            [z_wr_int_h]
0 MB            [z_wr_int_h]
0 MB            [z_wr_int_h]
0 MB            [z_wr_iss]
0 MB            [z_wr_iss]
0 MB            [z_wr_iss]
0 MB            [z_wr_iss_h]
0 MB            [z_wr_iss_h]
0 MB            [z_wr_iss_h]
0 MB            [z_wr_iss_h]
0 MB            [z_wr_iss_h]
0 MB            [z_zrele]
0 MB            [z_zvol]
1.75 MB         /sbin/agetty
1.875 MB                sort
2.25 MB         awk
2.5 MB          /usr/sbin/cron
2.90234 MB              /usr/sbin/chronyd
3.12891 MB              /usr/sbin/chronyd
3.625 MB                /sbin/rpcbind
3.75 MB         /usr/sbin/zed
4.39844 MB              /usr/lib/postfix/sbin/master
4.75 MB         ps
4.92188 MB              /usr/bin/dbus-daemon
5 MB            -bash
5.57812 MB              /lib/systemd/systemd-udevd
5.75 MB         /usr/sbin/smartd
6.15625 MB              (sd-pam)
6.25 MB         /lib/systemd/systemd-logind
6.5 MB          pickup
6.875 MB                qmgr
7.5 MB          sshd:
9 MB            tlsmgr
10.375 MB               /lib/systemd/systemd
10.75 MB                sshd:
10.875 MB               /lib/systemd/systemd-resolved
11.6836 MB              /sbin/init
15 MB           /usr/bin/python3
19.2656 MB              /usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-api
20.1836 MB              /usr/bin/prometheus-node-exporter
29.9727 MB              /usr/sbin/tailscaled
107.805 MB              /usr/bin/promtail
124.77 MB               /usr/sbin/corosync
295.562 MB              /lib/systemd/systemd-journald
300.969 MB              /usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-proxy

I'm not sure what could've possibly changed in the migration that would cause this. On my old server "used" RAM never went into the GBs.

Any help would be much appreciated.
 
Hi!
that's normal – moving to better hardware, i.e. faster, bigger memory will also use more memory. The kernel will use as much memory as there is available (for e.g. caching files). If this is a problem for you, this behavior can be adjusted by increasing the swappiness (edit /etc/sysctl.conf and add vm.swappiness = <value>, then apply using sysctl -p).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!