Can you please add UKSM into kernel?

D

Death_Master

New Member
Dec 6, 2011
4
0
1
Can you please add UKSM into kernel?
While some types of VMs can benefit from KSM it can be much better option to add UKSM so even containers and OS itself can benefit from RAM de-duplication.
Also UKSM is faster and more efficient.
 
Yes, it is much different.
Main idea is the same - memory de-duplication.
But it has some advantages over usual KSM(quoting the page):
  • Full system scan.
It automatically scans all user processes’ anonymous VMAs. Before UKSM, a process need to call KSM kernel API to submit its memory areas to KSM for scan. This makes many legacy software other than KVM can NOT benefit from KSM. Now, UKSM scans whole system applications including KVM. All programs benefit from UKSM without even knowing how it works.
  • Super quiet CPU usage. It automatically detects rich areas containing abundant duplicated pages. Rich areas are given a full scan speed. Poor areas are sampled at a reasonable speed with very low CPU consumption usually under 1%. Our benchmarks show that even for CPU intensive workloads, it has a negligible performance impact.
  • Ultra scan speed and CPU efficiency. A new hash algorithm is proposed. As a result, on a machine withCore(TM)2 Quad Q9300 CPU in 32-bit mode and 800MHZ DDR2 main memory, it can scan memory areas that does not contain duplicated pages at speed of 627MB/sec ~ 2445MB/sec and can merge duplicated areas at speed of 477MB/sec ~ 923MB/sec. For a bunch of busy workloads creating lots of duplicated pages, this means, with UKSM, you no long suffer from deadly swapping which is observed in KSM enabled setting.
  • Thrashing area avoidance. If a VM is constantly writing the same data to the duplicated pages, KSM suffer a problem of thrashing, i.e. the pages it merges will soon get copied again. In this situation, the CPU is purely wasted without having much memory saving. UKSM can perfectly avoid this situation by detecting the thrashing areas.
 
So, it is currently not merged into mainline kernel, which is usually a bad sign, because Linus may not want it. I normally would not use a manually patched kernel for a production machine anymore, maybe 15 years ago, but not anymore.
 
LnxBil said:
So, it is currently not merged into mainline kernel, which is usually a bad sign, because Linus may not want it. I normally would not use a manually patched kernel for a production machine anymore, maybe 15 years ago, but not anymore.
Click to expand...

Think this is a load of bs.

Before Proxmox VE had lxc, we had openvz right? That was patched in. Every Debian kernel is a patched kernel, every Fedora Kernel is patched, Centos kernel is patched, so this makes no sense to say such a thing.

Proxmox VE is a virtualization platform, the reason it has lxc or former openvz support, is because kvm as great as it is, its no where near as good as lxc when it comes to performance for linux hosts/containers.

Kvm can use the KSM system, Lxc cant really use it.

UKSM is a Ksm like system, build on the base code of Ksm, its used by many more people than you think, even Huawei contributes code and bug fixes for it, and even a few minor linux distributions has it included.

Including it in the kernel for pve, is not the same as enabling it at default.

I see no reasons at all to not include it, or have it as another kernel to have installed.
 
geekcamp said:
Before Proxmox VE had lxc, we had openvz right? That was patched in. Every Debian kernel is a patched kernel, every Fedora Kernel is patched, Centos kernel is patched, so this makes no sense to say such a thing.
Click to expand...

I'm talking about a manually patched kernel as in "did the patching and compiling myself". Maintaining and testing this is unbearable for individuals or very small companies like Proxmox. If Ubuntu does include it ... it'll be easy to include into Proxmox VE. Maybe try to convince the Ubuntu LTS Kernel team to include, maintain and test it.
 
Ah, sorry. Was thinking Thatcher YouTube where refering to the feature should not be included as it was not in Mainline kernel. Sorry for that.

Yes, it would be nice not to have to patch the kernel yourself, for something like This that grand a better experience for systems that have a memory constraint on em, like homelabs, or large scale systems.
 
I did a cursory google search for uksm, and here are my conclusions:

1. The original author is no longer involved (https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10).
2. The original author is still actively involved, or at least said so in March 2017.
https://www.phoronix.com/forums/for...ation-of-the-linux-kernel?p=936051#post936051. his last commit to the code was in July 2017 (https://github.com/dolohow/uksm/commits/master)
3. The code does not appear to be vetted or independently verified, doesnt have any security CVEs, nor has it been submitted or considered for kernel integration.

If Proxmox was irresponsible enough to integrate code of this quality into its codebase, I wouldnt suggest you would use it.
 
Too bad UKSM is not a more serious project. Eve-NG uses it where it works like a charm. Granted, Eve-NG usage is not close to production use cases, but it's still cool to run a lab with 15 vMX routers all of which have 2 GB of RAM available, and have the Eve-NG system only use a couple of GB of RAM in total.
 
There is still some discussion about mainlining this: https://github.com/dolohow/uksm/issues/41#issuecomment-926282376
I think this might need fulltime developer for one or two months to get into upstream. But still might be well worth it for all the large scale PVE/LXC deployments out there. Would be great if Proxmox developers would find some spare time and push this bit forward, since Proxmox is cutting edge "CT as VM" platform and no other project would see such vast benefit as PVE would from implementing this.
 
You must log in or register to reply here.
Top Bottom