KSM - Will it work for LXC?

harvie

Active Member
Apr 5, 2017
131
22
38
33
I was experimenting with KSM. I wonder if it can work on PVE with lots of LXC containers running same apps. Eg.: lots of apaches. I've enabled ksmtuned with threshold of 50%, it seems to run=1, but pages_shared:0 means it's not sharing and full_scans:0 probably means it didn't even tried to find the dedupable pages...

Any ideas?

/sys/kernel/mm/ksm/stable_node_dups:0
/sys/kernel/mm/ksm/max_page_sharing:256
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/stable_node_chains_prune_millisecs:2000
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/stable_node_chains:0
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/use_zero_pages:0
/sys/kernel/mm/ksm/pages_to_scan:1250
/sys/kernel/mm/ksm/sleep_millisecs:22
/sys/kernel/mm/ksm/run:1
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/pages_sharing:0
 

lhorace

Active Member
Oct 17, 2015
186
24
38
Hello there,

AFAIK, KSM is for Virtual Machines using Memory Balloon. Containers essentially share host memory via cgroups.

Cheers.
 

ertanerbek

Member
Mar 29, 2019
91
6
13
41
On container, system not upload any new kernel to memory, it is mean all module, driver already linked to container so why we need KSM ? Container use same kernel with source operation system
 

ertanerbek

Member
Mar 29, 2019
91
6
13
41
KSM is memory linked system, means if any same code stay on memory with diffrent memory address KSM was scan that and use one memory area for all and create link for each other.. For Container all container system was linked from source system on this memoment it is mean you not need KSM for container..


For clear understand KSM work as dedublication system. So if I have one file and create sembolic link on many diffrent area without copy so deduplication how can singularize my that file because I just have one file and many link...
 
Last edited:

lhorace

Active Member
Oct 17, 2015
186
24
38
Exactly, I posted links earlier pointing to documentation but they awaiting moderation, I'll quote relevant parts:

KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we're able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.
 

harvie

Active Member
Apr 5, 2017
131
22
38
33
One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.

Yes. I understand this. KSM should not be about VMs or CTs at all. It should detect all cases in which there are duplicate memory pages. That is exactly why i started this thread.

Because i can't get KSM to scan for duplicate pages at all.

full_scans:0 means that KSM didn't even tried to find the duplicate pages in the first place. And i don't know why, since run is 1.
 

harvie

Active Member
Apr 5, 2017
131
22
38
33
I think i've found possible culprit of this problem. KSM only deduplicates (merges) memory pages that were flagged with MADV_MERGEABLE flag using madvise() syscall. Recent QEMU versions are using madvise() to advise memory pages used by VMs to be merged.

KSM is available in mainline kernel. Besides of KSM there are 3rd party patches called UKSM and PKSM, which should work for whole system rather than just on pages advised for merging. However currently none of these patches were merged to upstream. I am not sure why. Probably there are some issues with their approach. Maybe they are just not mature yet.

I don't know. Maybe someone with more knowledge on this topic can prove me wrong or explain inner workings of these systems.

https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10
https://www.phoronix.com/scan.php?page=news_item&px=MTM0OTQ

Maybe we can try patching libc to add MADV_MERGEABLE flag to all memory upon it's allocation. Maybe even DLL injected libc wrapper can do this.
 
Last edited:

lhorace

Active Member
Oct 17, 2015
186
24
38
It should detect all cases in which there are duplicate memory pages

KSM is a memory-saving de-duplication feature, that merges anonymous (private) pages (not pagecache ones). Although it started this way, KSM is currently suitable for more than Virtual Machine use, as it can be useful to any application which generates many instances of the same data

Sure.

I think i've found possible culprit of this problem. KSM only deduplicates (merges) memory pages that were flagged with MADV_MERGEABLE flag using madvise() syscall. Recent QEMU versions are using madvise() to advise memory pages used by VMs to be merged.

It should work out of the box since Proxmox has support for it. Why it isn't working, you found part of the reason. And it's likely proxmox has support for only Qemu processes.

Maybe we can try patching libc to add MADV_MERGEABLE flag to all memory upon it's allocation. Maybe even DLL injected libc wrapper can do this.

It's likely the support is already there.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!