I have a host with two NUMA nodes, and I would like to create a VM with two NUMA nodes, with the cores from each VM node pinned to cores on the corresponding host node.
So far, I've found the numa[n] options. But I'm a bit unclear about their usage, and the docs are pretty sparse. I tried setting:
I thought this would give pin the 62 cores of VM node0 to host cores 2-63, and the 62 cores of VM node1 to host cores 66-127. However this makes the VM start fail with "kvm: -numa node,nodeid=1,cpus=66-127,memdev=ram-node1: CPU index (124) should be smaller than maxcpus (124)". So it seems the cpus= setting for the numa[n] option must be referencing VM cores rather than host cores. So, okay, I guess it's for mapping VM cores to VM numa node. And the hostnodes= options then specifies the host node to use for the memory allocation for that VM node, I assume. But does it also implicitly pin the VM node0 CPUs to host node0 CPUs?
I can use cpuaffinity to pin the VM to a set of host cores, but AFAICT it's just pinning the qemu process as a whole, so I assume there is no guarantee that qemu CPU threads won't be scheduled on the wrong host node?
Ideally I'd like to pin each VM core individually to a specific host core, to minimize cache misses. But I can't find any way to do that.
Any thoughts or ideas?
So far, I've found the numa[n] options. But I'm a bit unclear about their usage, and the docs are pretty sparse. I tried setting:
Code:
sockets: 2
cores: 62
numa: 1
numa0: cpus=2-63,hostnodes=0,memory=210944,policy=bind
numa1: cpus=66-127,hostnodes=1,memory=210944,policy=bindI thought this would give pin the 62 cores of VM node0 to host cores 2-63, and the 62 cores of VM node1 to host cores 66-127. However this makes the VM start fail with "kvm: -numa node,nodeid=1,cpus=66-127,memdev=ram-node1: CPU index (124) should be smaller than maxcpus (124)". So it seems the cpus= setting for the numa[n] option must be referencing VM cores rather than host cores. So, okay, I guess it's for mapping VM cores to VM numa node. And the hostnodes= options then specifies the host node to use for the memory allocation for that VM node, I assume. But does it also implicitly pin the VM node0 CPUs to host node0 CPUs?
I can use cpuaffinity to pin the VM to a set of host cores, but AFAICT it's just pinning the qemu process as a whole, so I assume there is no guarantee that qemu CPU threads won't be scheduled on the wrong host node?
Ideally I'd like to pin each VM core individually to a specific host core, to minimize cache misses. But I can't find any way to do that.
Any thoughts or ideas?
Last edited:
