KSM - Will it work for LXC?

[harvie]

I was experimenting with KSM. I wonder if it can work on PVE with lots of LXC containers running same apps. Eg.: lots of apaches. I've enabled ksmtuned with threshold of 50%, it seems to run=1, but pages_shared:0 means it's not sharing and full_scans:0 probably means it didn't even tried to find the dedupable pages...

Any ideas?

/sys/kernel/mm/ksm/stable_node_dups:0
/sys/kernel/mm/ksm/max_page_sharing:256
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/stable_node_chains_prune_millisecs:2000
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/stable_node_chains:0
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/use_zero_pages:0
/sys/kernel/mm/ksm/pages_to_scan:1250
/sys/kernel/mm/ksm/sleep_millisecs:22
/sys/kernel/mm/ksm/run:1
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/pages_sharing:0

 

[lhorace]

Hello there,

AFAIK, KSM is for Virtual Machines using Memory Balloon. Containers essentially share host memory via cgroups.

Cheers.

 

[ertanerbek]

On container, system not upload any new kernel to memory, it is mean all module, driver already linked to container so why we need KSM ? Container use same kernel with source operation system

 

[lhorace]

why we need KSM

AFAIK, KSM is for Virtual Machines using Memory Balloon.

 

[lhorace]

Have a look at https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_memory and https://pve.proxmox.com/wiki/Dynamic_Memory_Management#KSM

 

[ertanerbek]

KSM is memory linked system, means if any same code stay on memory with diffrent memory address KSM was scan that and use one memory area for all and create link for each other.. For Container all container system was linked from source system on this memoment it is mean you not need KSM for container..


For clear understand KSM work as dedublication system. So if I have one file and create sembolic link on many diffrent area without copy so deduplication how can singularize my that file because I just have one file and many link...

 

[lhorace]

Exactly, I posted links earlier pointing to documentation but they awaiting moderation, I'll quote relevant parts:

KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we're able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.

 

[harvie]

One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.

Yes. I understand this. KSM should not be about VMs or CTs at all. It should detect all cases in which there are duplicate memory pages. That is exactly why i started this thread.

Because i can't get KSM to scan for duplicate pages at all.

full_scans:0 means that KSM didn't even tried to find the duplicate pages in the first place. And i don't know why, since run is 1.

 

[harvie]

I think i've found possible culprit of this problem. KSM only deduplicates (merges) memory pages that were flagged with MADV_MERGEABLE flag using madvise() syscall. Recent QEMU versions are using madvise() to advise memory pages used by VMs to be merged.

KSM is available in mainline kernel. Besides of KSM there are 3rd party patches called UKSM and PKSM, which should work for whole system rather than just on pages advised for merging. However currently none of these patches were merged to upstream. I am not sure why. Probably there are some issues with their approach. Maybe they are just not mature yet.

I don't know. Maybe someone with more knowledge on this topic can prove me wrong or explain inner workings of these systems.

https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10
https://www.phoronix.com/scan.php?page=news_item&px=MTM0OTQ

Maybe we can try patching libc to add MADV_MERGEABLE flag to all memory upon it's allocation. Maybe even DLL injected libc wrapper can do this.

 

[lhorace]

It should detect all cases in which there are duplicate memory pages

KSM is a memory-saving de-duplication feature, that merges anonymous (private) pages (not pagecache ones). Although it started this way, KSM is currently suitable for more than Virtual Machine use, as it can be useful to any application which generates many instances of the same data

Sure.

I think i've found possible culprit of this problem. KSM only deduplicates (merges) memory pages that were flagged with MADV_MERGEABLE flag using madvise() syscall. Recent QEMU versions are using madvise() to advise memory pages used by VMs to be merged.

It should work out of the box since Proxmox has support for it. Why it isn't working, you found part of the reason. And it's likely proxmox has support for only Qemu processes.

Maybe we can try patching libc to add MADV_MERGEABLE flag to all memory upon it's allocation. Maybe even DLL injected libc wrapper can do this.

It's likely the support is already there.