Can you please add UKSM into kernel?
- Thread starter Death_Master
- Start date
Page does currently not load for me, so is this somehow different than the "ordinary" KSM that is already enabled?
https://pve.proxmox.com/wiki/Dynamic_Memory_Management#KSM
https://pve.proxmox.com/wiki/Dynamic_Memory_Management#KSM
Yes, it is much different.
Main idea is the same - memory de-duplication.
But it has some advantages over usual KSM(quoting the page):
Main idea is the same - memory de-duplication.
But it has some advantages over usual KSM(quoting the page):
- Full system scan.
- Super quiet CPU usage. It automatically detects rich areas containing abundant duplicated pages. Rich areas are given a full scan speed. Poor areas are sampled at a reasonable speed with very low CPU consumption usually under 1%. Our benchmarks show that even for CPU intensive workloads, it has a negligible performance impact.
- Ultra scan speed and CPU efficiency. A new hash algorithm is proposed. As a result, on a machine withCore(TM)2 Quad Q9300 CPU in 32-bit mode and 800MHZ DDR2 main memory, it can scan memory areas that does not contain duplicated pages at speed of 627MB/sec ~ 2445MB/sec and can merge duplicated areas at speed of 477MB/sec ~ 923MB/sec. For a bunch of busy workloads creating lots of duplicated pages, this means, with UKSM, you no long suffer from deadly swapping which is observed in KSM enabled setting.
- Thrashing area avoidance. If a VM is constantly writing the same data to the duplicated pages, KSM suffer a problem of thrashing, i.e. the pages it merges will soon get copied again. In this situation, the CPU is purely wasted without having much memory saving. UKSM can perfectly avoid this situation by detecting the thrashing areas.
So, it is currently not merged into mainline kernel, which is usually a bad sign, because Linus may not want it. I normally would not use a manually patched kernel for a production machine anymore, maybe 15 years ago, but not anymore.
Think this is a load of bs.
Before Proxmox VE had lxc, we had openvz right? That was patched in. Every Debian kernel is a patched kernel, every Fedora Kernel is patched, Centos kernel is patched, so this makes no sense to say such a thing.
Proxmox VE is a virtualization platform, the reason it has lxc or former openvz support, is because kvm as great as it is, its no where near as good as lxc when it comes to performance for linux hosts/containers.
Kvm can use the KSM system, Lxc cant really use it.
UKSM is a Ksm like system, build on the base code of Ksm, its used by many more people than you think, even Huawei contributes code and bug fixes for it, and even a few minor linux distributions has it included.
Including it in the kernel for pve, is not the same as enabling it at default.
I see no reasons at all to not include it, or have it as another kernel to have installed.
I'm talking about a manually patched kernel as in "did the patching and compiling myself". Maintaining and testing this is unbearable for individuals or very small companies like Proxmox. If Ubuntu does include it ... it'll be easy to include into Proxmox VE. Maybe try to convince the Ubuntu LTS Kernel team to include, maintain and test it.
Ah, sorry. Was thinking Thatcher YouTube where refering to the feature should not be included as it was not in Mainline kernel. Sorry for that.
Yes, it would be nice not to have to patch the kernel yourself, for something like This that grand a better experience for systems that have a memory constraint on em, like homelabs, or large scale systems.
Yes, it would be nice not to have to patch the kernel yourself, for something like This that grand a better experience for systems that have a memory constraint on em, like homelabs, or large scale systems.
I did a cursory google search for uksm, and here are my conclusions:
1. The original author is no longer involved (https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10).
2. The original author is still actively involved, or at least said so in March 2017.
https://www.phoronix.com/forums/for...ation-of-the-linux-kernel?p=936051#post936051. his last commit to the code was in July 2017 (https://github.com/dolohow/uksm/commits/master)
3. The code does not appear to be vetted or independently verified, doesnt have any security CVEs, nor has it been submitted or considered for kernel integration.
If Proxmox was irresponsible enough to integrate code of this quality into its codebase, I wouldnt suggest you would use it.
1. The original author is no longer involved (https://www.phoronix.com/scan.php?page=news_item&px=UKSM-Linux-4.10).
2. The original author is still actively involved, or at least said so in March 2017.
https://www.phoronix.com/forums/for...ation-of-the-linux-kernel?p=936051#post936051. his last commit to the code was in July 2017 (https://github.com/dolohow/uksm/commits/master)
3. The code does not appear to be vetted or independently verified, doesnt have any security CVEs, nor has it been submitted or considered for kernel integration.
If Proxmox was irresponsible enough to integrate code of this quality into its codebase, I wouldnt suggest you would use it.
Too bad UKSM is not a more serious project. Eve-NG uses it where it works like a charm. Granted, Eve-NG usage is not close to production use cases, but it's still cool to run a lab with 15 vMX routers all of which have 2 GB of RAM available, and have the Eve-NG system only use a couple of GB of RAM in total.
There is still some discussion about mainlining this: https://github.com/dolohow/uksm/issues/41#issuecomment-926282376
I think this might need fulltime developer for one or two months to get into upstream. But still might be well worth it for all the large scale PVE/LXC deployments out there. Would be great if Proxmox developers would find some spare time and push this bit forward, since Proxmox is cutting edge "CT as VM" platform and no other project would see such vast benefit as PVE would from implementing this.
I think this might need fulltime developer for one or two months to get into upstream. But still might be well worth it for all the large scale PVE/LXC deployments out there. Would be great if Proxmox developers would find some spare time and push this bit forward, since Proxmox is cutting edge "CT as VM" platform and no other project would see such vast benefit as PVE would from implementing this.
Anyway, i went ahead and suggested this in pve bugzilla: https://bugzilla.proxmox.com/show_bug.cgi?id=3637