Swap usage is out of control on kernel 6.8.12-1-pve

[Kingneutron]

I have 2 swap locations defined, /dev/dm-0 is 2GB from the installer and /dev/zram0 is 256MB

Swappiness is set to 1 and swap is consistently filling up after midnight / backups.

I did not have this problem on kernel 5.15.136-1-pve (very minimal swap usage after weeks of uptime) and only recently upgraded proxmox kernel after they recently fixed an issue with my 10Gbit NICs.

Any ideas? I have a community subscription license.


cat /proc/sys/vm/swappiness
1


Other tweaks in place:

# hopefully better multitasking I/O performance
echo 20 > /proc/sys/vm/dirty_ratio

# Try to keep at least 100MB of free RAM at all times
echo 100000 > /proc/sys/vm/min_free_kbytes

# Default 100 - try more aggressively to reclaim inodes, etc from cache
echo 160 > /proc/sys/vm/vfs_cache_pressure


System details:
8-core Intel Atom C3758 @ 2.20GHz
32GB RAM

Qotom firewall appliance Q20332G9
https://www.amazon.com/gp/product/B0CJLK9GZV


pveversion
pve-manager/8.2.5/12c0a59769080547 (running kernel: 6.8.12-1-pve)


free -h

total        used        free      shared  buff/cache   available
Mem:            31Gi        20Gi       2.2Gi       108Mi       9.0Gi        10Gi
Swap:          2.2Gi       2.2Gi        57Mi