Can you please add UKSM into kernel?

Death_Master

New Member
Dec 6, 2011
4
0
1
Can you please add UKSM into kernel?
While some types of VMs can benefit from KSM it can be much better option to add UKSM so even containers and OS itself can benefit from RAM de-duplication.
Also UKSM is faster and more efficient.
 

Death_Master

New Member
Dec 6, 2011
4
0
1
Yes, it is much different.
Main idea is the same - memory de-duplication.
But it has some advantages over usual KSM(quoting the page):
  • Full system scan.
It automatically scans all user processes’ anonymous VMAs. Before UKSM, a process need to call KSM kernel API to submit its memory areas to KSM for scan. This makes many legacy software other than KVM can NOT benefit from KSM. Now, UKSM scans whole system applications including KVM. All programs benefit from UKSM without even knowing how it works.
  • Super quiet CPU usage. It automatically detects rich areas containing abundant duplicated pages. Rich areas are given a full scan speed. Poor areas are sampled at a reasonable speed with very low CPU consumption usually under 1%. Our benchmarks show that even for CPU intensive workloads, it has a negligible performance impact.
  • Ultra scan speed and CPU efficiency. A new hash algorithm is proposed. As a result, on a machine withCore(TM)2 Quad Q9300 CPU in 32-bit mode and 800MHZ DDR2 main memory, it can scan memory areas that does not contain duplicated pages at speed of 627MB/sec ~ 2445MB/sec and can merge duplicated areas at speed of 477MB/sec ~ 923MB/sec. For a bunch of busy workloads creating lots of duplicated pages, this means, with UKSM, you no long suffer from deadly swapping which is observed in KSM enabled setting.
  • Thrashing area avoidance. If a VM is constantly writing the same data to the duplicated pages, KSM suffer a problem of thrashing, i.e. the pages it merges will soon get copied again. In this situation, the CPU is purely wasted without having much memory saving. UKSM can perfectly avoid this situation by detecting the thrashing areas.
 

LnxBil

Famous Member
Feb 21, 2015
6,059
740
133
Germany
So, it is currently not merged into mainline kernel, which is usually a bad sign, because Linus may not want it. I normally would not use a manually patched kernel for a production machine anymore, maybe 15 years ago, but not anymore.
 

geekcamp

New Member
Aug 20, 2013
7
0
1
So, it is currently not merged into mainline kernel, which is usually a bad sign, because Linus may not want it. I normally would not use a manually patched kernel for a production machine anymore, maybe 15 years ago, but not anymore.

Think this is a load of bs.

Before Proxmox VE had lxc, we had openvz right? That was patched in. Every Debian kernel is a patched kernel, every Fedora Kernel is patched, Centos kernel is patched, so this makes no sense to say such a thing.

Proxmox VE is a virtualization platform, the reason it has lxc or former openvz support, is because kvm as great as it is, its no where near as good as lxc when it comes to performance for linux hosts/containers.

Kvm can use the KSM system, Lxc cant really use it.

UKSM is a Ksm like system, build on the base code of Ksm, its used by many more people than you think, even Huawei contributes code and bug fixes for it, and even a few minor linux distributions has it included.

Including it in the kernel for pve, is not the same as enabling it at default.

I see no reasons at all to not include it, or have it as another kernel to have installed.
 

LnxBil

Famous Member
Feb 21, 2015
6,059
740
133
Germany
Before Proxmox VE had lxc, we had openvz right? That was patched in. Every Debian kernel is a patched kernel, every Fedora Kernel is patched, Centos kernel is patched, so this makes no sense to say such a thing.

I'm talking about a manually patched kernel as in "did the patching and compiling myself". Maintaining and testing this is unbearable for individuals or very small companies like Proxmox. If Ubuntu does include it ... it'll be easy to include into Proxmox VE. Maybe try to convince the Ubuntu LTS Kernel team to include, maintain and test it.
 

geekcamp

New Member
Aug 20, 2013
7
0
1
Ah, sorry. Was thinking Thatcher YouTube where refering to the feature should not be included as it was not in Mainline kernel. Sorry for that.

Yes, it would be nice not to have to patch the kernel yourself, for something like This that grand a better experience for systems that have a memory constraint on em, like homelabs, or large scale systems.
 

alexskysilk

Well-Known Member
Oct 16, 2015
728
86
48
Chatsworth, CA
www.skysilk.com
I did a cursory google search for uksm, and here are my conclusions:

1. The original author is no longer involved (https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10).
2. The original author is still actively involved, or at least said so in March 2017.
https://www.phoronix.com/forums/for...ation-of-the-linux-kernel?p=936051#post936051. his last commit to the code was in July 2017 (https://github.com/dolohow/uksm/commits/master)
3. The code does not appear to be vetted or independently verified, doesnt have any security CVEs, nor has it been submitted or considered for kernel integration.

If Proxmox was irresponsible enough to integrate code of this quality into its codebase, I wouldnt suggest you would use it.
 

dkong

New Member
Aug 13, 2017
16
0
1
36
Too bad UKSM is not a more serious project. Eve-NG uses it where it works like a charm. Granted, Eve-NG usage is not close to production use cases, but it's still cool to run a lab with 15 vMX routers all of which have 2 GB of RAM available, and have the Eve-NG system only use a couple of GB of RAM in total.
 

harvie

Active Member
Apr 5, 2017
130
21
38
32
There is still some discussion about mainlining this: https://github.com/dolohow/uksm/issues/41#issuecomment-926282376
I think this might need fulltime developer for one or two months to get into upstream. But still might be well worth it for all the large scale PVE/LXC deployments out there. Would be great if Proxmox developers would find some spare time and push this bit forward, since Proxmox is cutting edge "CT as VM" platform and no other project would see such vast benefit as PVE would from implementing this.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!