Taskset CPU pinning

dzdz

New Member
May 31, 2022
4
0
1
I am running proxmox 7.1 on a Ryzen 5950x with 5 vms, and would like to pin each of the respective vms to particular cores.

After reading this post about CPU pinning, I have tried to use taskset to do so. However, when running taskset --cpu-list --all-tasks --pid 0-5 $vm-pid as suggested in the post, I receive the following error:

pid <pid> current affinity list: 0-31
pid <pid> new affinity list: 0-5
pid <pid> current affinity list: 0-31
pid <pid> new affinity list: 0-5
pid <pid> current affinity list: 0-31
taskset: failed to set pid <pid> affinity: Invalid argument

Upon inspection via ps -Tp <pid>, the pid in question has iou-wrk-<pid> threads causing taskset to fail, leaving me stumped on how to proceed.

Does anybody have any suggestions?
 

dzdz

New Member
May 31, 2022
4
0
1
@LnxBil I'm working on a high performance computing application and have noticed that the CPU scheduler seems to be negatively affecting performance. Namely, if vms are allotted cores such that the CPU is oversubscribed, performance decreases, but if the same number of vms are allotted cores such that the CPU is undersubscribed, performance increases. I'm trying to take my experiment a step further to see if CPU pegging offers further performance benefits.
 

_gabriel

Member
Mar 30, 2021
101
12
18
37
how many cores per VM do you allocate ?
afaik, if all VMs has 32 cores, each vm wait for their run (sorry for my english)
 

LnxBil

Famous Member
Feb 21, 2015
6,305
779
163
Saarland, Germany
@LnxBil I'm working on a high performance computing application and have noticed that the CPU scheduler seems to be negatively affecting performance. Namely, if vms are allotted cores such that the CPU is oversubscribed, performance decreases, but if the same number of vms are allotted cores such that the CPU is undersubscribed, performance increases. I'm trying to take my experiment a step further to see if CPU pegging offers further performance benefits.
In my experiments, this is not a big deal. Having the correct NUMA node (numa pinning if you will) and having hugepages available will significantly improve the performance (>10%), 1 GB hugepages especially if you have a lot of RAM (>256 GB).
 

dzdz

New Member
May 31, 2022
4
0
1
how many cores per VM do you allocate ?
afaik, if all VMs has 32 cores, each vm wait for their run (sorry for my english)
AMD 5950X has 16 cores, 32 threads. I previously assigned each of my 5 vms 8 cores, but saw performance issues. Dropping to 6 cores improved performance.
 

dzdz

New Member
May 31, 2022
4
0
1
In my experiments, this is not a big deal. Having the correct NUMA node (numa pinning if you will) and having hugepages available will significantly improve the performance (>10%), 1 GB hugepages especially if you have a lot of RAM (>256 GB).
Very interesting! Another avenue to pursue. Thank you for your input!
 

1pawn

New Member
Nov 26, 2021
2
0
1
45
Hi guys, great discussion here. My server (powerdge T630) has 2 CPUs, for a total of 32 threads. It has 256GB ram (16x16GB). That's a bit overkill for my home needs. With a couple of VM running, consumption is around 130W. I was wondering if it's possible, thanks to CPU pinning and some energy efficient management, to put the 2nd CPU to sleep, and have it to work only if needed.
For example, I wanted to use CPU pinning on some essential services (like pfsense firewall, truenas) to the CPU1, and leave some "extra" things (like gaming/windows VM with GPu pass-through) to CPU 2.
I already installed the essential PCIe cards to the PCIe slots connected to CPU1 (like network/nvme), and the "extra" ones to CPU2 (like the GPU).
I'm now missing the software magic :) Anybody tried this already?
If CPU pinning is not the answer, and I instead manage to find a way to temporarily put CPU2 to sleep, would Proxmox automatically run all services/VM from CPU1?
Thanks a lot!
 

LnxBil

Famous Member
Feb 21, 2015
6,305
779
163
Saarland, Germany
With a couple of VM running, consumption is around 130W.
That is not much for a server. Try running a CPU benchmark on all CPUs for a e.g. 15 minutes and look at the numbers then.

to put the 2nd CPU to sleep, and have it to work only if needed.
You cannot put a CPU to sleep. If it does not run anything it will still manage the ram, it will go into normal idle mode and uses its normal idle power. If you want better energy efficiency, unplug a CPU or go with special low-power systems.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!