Modifications to make on large node? (1TB RAM)

Mirkat

New Member
Jun 17, 2021
1
0
1
30
I am wondering what is some of the changes I should make to KSM, swap and anything else you guys suggest for very large RAM nodes?

Currently I have read I should make the following changes to KSM

Code:
KSM_THRES_COEF=10
KSM_SLEEP_MSEC=10

The only problem I have found with this so far is the total "KSM Sharing" stays around 100GB but with the default value it goes as high as 300GB should I switch that back?

The server also has ZFS set to 20GB min 40GB max

Also I have changed swap to enable at <10% RAM

Code:
vm.swappiness=10

I have been running tests with swap. I eat 200GB of the nodes RAM during a test and found that

Code:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     65815 21.5  5.6 69967176 59383812 ?   Sl   Jun10 2164:55 /usr/bin/kvm -id 16

This virtual machines RSS memory was lowered which is great as it's reserving 60GB RAM but was only currently using about 3GB on the graphs.

Code:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     65815 21.4  4.5 69967176 47985040 ?   Sl   Jun10 2166:10 /usr/bin/kvm -id 16

What happens to this 199GB swap now it's been hovering around that now for hours

Few other changes added are

Code:
vm.min_free_kbytes=524288
vm.nr_hugepages=72
vm.max_map_count=262144
vm.overcommit_memory = 1

2955c912108ddffbdb2d91bf05165121.png

Before/After eating RAM



a5eb8a51c958e75819f5ee1524880478.png
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!